Fixed, improved, and unified Docker environment #5474
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -47,4 +47,5 @@ wandb | |
| /docker-compose.yaml | ||
| /docker-compose.yml | ||
| /Dockerfile | ||
| .dockerignore | ||
| .env | ||
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -1,21 +1,22 @@ | ||||||||
| FROM ubuntu:22.04 | ||||||||
|
|
||||||||
| ARG GPU_CHOICE=${GPU_CHOICE:-A} # A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU) | ||||||||
|
There was a problem hiding this comment. This and other Why are they using var assignment?
Suggested change
You can override that default with |
||||||||
| ARG TORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX}" | ||||||||
| ARG BUILD_EXTENSIONS="${BUILD_EXTENSIONS:-}" | ||||||||
| ARG APP_UID="${APP_UID:-6972}" | ||||||||
| ARG APP_GID="${APP_GID:-6972}" | ||||||||
|
|
||||||||
| WORKDIR /builder | ||||||||
|
There was a problem hiding this comment. What is the point of this directive? You don't use |
||||||||
| RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,rw \ | ||||||||
| apt update && \ | ||||||||
| apt install --no-install-recommends -y git vim build-essential python3-dev pip bash curl && \ | ||||||||
| rm -rf /var/lib/apt/lists/* | ||||||||
| WORKDIR /home/app/ | ||||||||
| RUN git clone https://github.com/oobabooga/text-generation-webui.git | ||||||||
|
There was a problem hiding this comment. A Since this is already what the image has been doing, I'd defer changing this until a follow-up PR, but you should build from the repo with the repo root as the build context. The benefit is you can go to any tag/commit of the repo from that point onwards and build the image if needed. It's better for automation in CI too. There was a problem hiding this comment. You've got a
Suggested change
Remove the other two |
||||||||
| WORKDIR /home/app/text-generation-webui | ||||||||
| RUN USE_CUDA118=FALSE LAUNCH_AFTER_INSTALL=FALSE INSTALL_EXTENSIONS=TRUE ./start_linux.sh --verbose | ||||||||
| COPY CMD_FLAGS.txt /home/app/text-generation-webui/ | ||||||||
| EXPOSE ${CONTAINER_PORT:-7860} ${CONTAINER_API_PORT:-5000} ${CONTAINER_API_STREAM_PORT:-5005} | ||||||||
| ENV HOME=/home/app/text-generation-webui | ||||||||
| # set umask to ensure group read / write at runtime | ||||||||
| CMD umask 0002 && ./start_linux.sh $CLI_ARGS | ||||||||
|
There was a problem hiding this comment.
Suggested change
Now In |
||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,42 +4,46 @@ services: | |
| build: | ||
| context: . | ||
| args: | ||
| GPU_CHOICE: B # AMD | ||
|
|
||
| # Requirements file to use: | ||
| # | GPU | CPU | requirements file to use | | ||
| # | -------- | ------------- | ---------------------------------- | | ||
| # | NVIDIA | has AVX2 | `requirements.txt` | | ||
| # | NVIDIA | no AVX2 | `requirements_noavx2.txt` | | ||
| # | AMD | has AVX2 | `requirements_amd.txt` | | ||
| # | AMD | no AVX2 | `requirements_amd_noavx2.txt` | | ||
| # | CPU only | has AVX2 | `requirements_cpu_only.txt` | | ||
| # | CPU only | no AVX2 | `requirements_cpu_only_noavx2.txt` | | ||
| # | Apple | Intel | `requirements_apple_intel.txt` | | ||
| # | Apple | Apple Silicon | `requirements_apple_silicon.txt` | | ||
| # Default: requirements.txt` | ||
| # BUILD_REQUIREMENTS: requirements.txt | ||
|
|
||
| # Extension requirements to build: | ||
| BUILD_EXTENSIONS: ${BUILD_EXTENSIONS:-} | ||
|
|
||
| # specify which cuda version your card supports: https://developer.nvidia.com/cuda-gpus | ||
| TORCH_CUDA_ARCH_LIST: ${TORCH_CUDA_ARCH_LIST:-7.5} | ||
| APP_GID: ${APP_GID:-6972} | ||
| APP_UID: ${APP_UID:-6972} | ||
|
Comment on lines
+28
to
+29
There was a problem hiding this comment. Are these actually used for anything? Nothing appears to reference them: https://github.com/search?q=repo%3Aoobabooga%2Ftext-generation-webui%20APP_UID&type=code If they're only meant to change the runtime user, this should just be done with the standard |
||
| container_name: text-generation-webui | ||
| env_file: .env | ||
| hostname: text-generation-webui | ||
|
|
||
| user: "${APP_RUNTIME_UID:-6972}:${APP_RUNTIME_GID:-6972}" | ||
| ports: | ||
| - "${HOST_PORT:-7860}:${CONTAINER_PORT:-7860}" | ||
| - "${HOST_API_PORT:-5000}:${CONTAINER_API_PORT:-5000}" | ||
| restart: unless-stopped | ||
| stdin_open: true | ||
| group_add: | ||
| - video | ||
| tty: true | ||
| ipc: host | ||
| devices: | ||
| - /dev/kfd | ||
| - /dev/dri | ||
| cap_add: | ||
| - SYS_PTRACE | ||
| security_opt: | ||
| - seccomp=unconfined | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would you want to ignore a
.dockerignore?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually... what is with ignoring these other Docker files prior to this PR? 🤔