- uses model from https://huggingface.co/SotirisLegkas/multi-head-xlm-xl-tokens-38
# run
tira-run \
--input-directory "$PWD/valueeval24/test" \
--output-directory "$PWD/output" \
--image webis/valueeval24-hierocles-of-alexandria:1.0.0
# or
docker run --rm \
-v "$PWD/valueeval24/test:/dataset" -v "$PWD/output:/output" \
webis/valueeval24-hierocles-of-alexandria:1.0.0
# view results
cat output/run.tsv
A local inference server can be started from the same Docker-Image using:
PORT=8001
docker run --rm -it --init \
-v "$PWD/logs:/logs" \
-p $PORT:$PORT \
--entrypoint tira-run-inference-server \
webis/valueeval24-hierocles-of-alexandria:1.0.0 \
--script /predict.py --port $PORT
# or, for zero-shot version
docker run --rm -it --init \
-v "$PWD/logs:/logs" \
-p $PORT:$PORT \
-e HOA_ZERO_SHOT="True" \
--entrypoint tira-run-inference-server \
webis/valueeval24-hierocles-of-alexandria:1.0.0 \
--script /predict.py --port $PORT
docker run --rm -it --init -v "$PWD/logs:/logs" -p 8001:8001 -e HOA_ZERO_SHOT="True" --entrypoint tira-run-inference-server valueeval24-hierocles-of-alexandria:1.0.0 --script /predict.py --port 8001
Exemplary request for a server running on localhost:8001 are
# POST (JSON list as payload)
curl -X POST -H "application/json" \
-d "[{\"Text\": \"element 1\", \"language\": \"EN\"}, {\"Text\": \"element 2\", \"language\": \"EN\"}]" \
localhost:8001
and
# GET (JSON object string(s) passed to the 'payload' parameter)
curl "localhost:8001?payload=\"element+1\"&payload=\"element+2\""
The possible values for language are EN
, EL
, DE
, TR
, FR
, BG
, HE
, IT
, NL
.
Please note that GET-request are currently only possible for language EN
.
Both webis/valueeval24-hierocles-of-alexandria:1.0.0
and webis/valueeval24-hierocles-of-alexandria:1.0.0-no-model
are available on dockerhub.
The webis/valueeval24-hierocles-of-alexandria:1.0.0-no-model
image is much smaller, but requires you to:
- use
download_model_and_tokenizer.py
separately - mount the created
models
andtokenizer
directories into the containers root directory
# build stand-alone image
docker build -f Dockerfile -t webis/valueeval24-hierocles-of-alexandria:1.0.0 .
# build image without models
DOCKER_BUILDKIT=1 docker build -f Dockerfile \
-t webis/valueeval24-hierocles-of-alexandria:1.0.0-no-model \
--build-arg include_models=false .
Running the image in our Webis cluster on a GPU.
srun --container-writable --mem=32g --cpus-per-task=8 --gres=gpu:3g.20gb:1 --container-image=webis/valueeval24-hierocles-of-alexandria:1.0.0-no-model \
/bin/bash -c "ln -s /mnt/ceph/storage/data-in-production/data-research/computational-ethics/valueeval/hierocles-of-alexandria24/models/SotirisLegkas /models/SotirisLegkas;ln -s /mnt/ceph/storage/data-in-production/data-research/computational-ethics/valueeval/hierocles-of-alexandria24/tokenizer/SotirisLegkas /tokenizer/SotirisLegkas;ln -s /mnt/ceph/storage/data-in-production/data-research/computational-ethics/valueeval/hierocles-of-alexandria24/logs;WORLD_SIZE=1 RANK=0 MASTER_PORT=8788 MASTER_ADDR=localhost tira-run-inference-server --script /predict.py --port 8787"