Fixed, improved, and unified Docker environment #5474

WolframRavenwolf · 2024-02-09T20:08:28Z

Checklist:

I have read the Contributing guidelines.

Fixed and improved Docker environment

Unified Dockerfile:
- Moved GPU_CHOICE from RUN to ARG instruction so all the Dockerfiles could be unified into a single Dockerfile
- Added CLI_ARGS to start_linux.sh command (very important fix, without it CLI_ARGS are ignored)
- Set HOME as global instead of process-specific envvar (so it's also set when entering the container manually)
- Removed redundant WORKDIR (we've already changed into it)
- Deleted old Dockerfiles (now obsolete because of the unified Dockerfile)
Composefiles:
- Fixed APP_UID variable default (added missing colon)
- Added sensible container_name ("text-generation-webui" instead of default "text-generation-webui-text-generation-webui-1")
- Added sensible hostname ("text-generation-webui" instead of random ID)
- Added restart policy "unless-stopped" (so container can be kept running or stopped properly when starting detached)
Envfile-Example:
- Added PYTORCH_KERNEL_CACHE_PATH pointing at cache dir to fix "UserWarning: Specified kernel cache directory could not be created! This disables kernel caching."
- Added APP_RUNTIME_UID to APP_RUNTIME_GID (so you can not only define the group, but also the user for access to mounted volumes)
Docs:
- Create symlinks (fixed) and missing dirs (so they won't be created by Docker as root)
.gitignore
- Added .dockerignore

- Moved GPU_CHOICE from RUN to ARG instruction so all the Dockerfiles could be unified into a single Dockerfile - Added CLI_ARGS to start_linux.sh command (very important fix, without it CLI_ARGS are ignored) - Set HOME as global instead of process-specific envvar (so it's also set when entering the container manually) - Removed redundant WORKDIR (we've already changed into it)

Added PYTORCH_KERNEL_CACHE_PATH pointing at cache dir to fix "UserWarning: Specified kernel cache directory could not be created! This disables kernel caching."

Removed trailing slashes from cache paths for consistency

Create symlinks and missing dirs

Added APP_RUNTIME_UID to APP_RUNTIME_GID (so you can not only define the group, but also the user for access to mounted volumes)

Added .dockerignore to .gitignore

jtara1 · 2024-02-19T02:56:26Z

Tested and it works for me. I can now define and use CLI_ARGS="--listen-host 0.0.0.0 --listen" as an env var set in my .env

oobabooga · 2024-07-20T22:08:25Z

Could you merge the dev branch?

polarathene

Some feedback to consider.

While I've not looked at the the files in full there's likely a bunch of duplication (at least from the repeated diffs still present) that could be unified to single files.

polarathene · 2024-08-17T22:23:20Z

.gitignore

@@ -47,4 +47,5 @@ wandb
 /docker-compose.yaml
 /docker-compose.yml
 /Dockerfile
+.dockerignore


Why would you want to ignore a .dockerignore?

Actually... what is with ignoring these other Docker files prior to this PR? 🤔

polarathene · 2024-08-17T22:32:58Z

docker/Dockerfile

 FROM ubuntu:22.04
-WORKDIR /builder
+
+ARG GPU_CHOICE=${GPU_CHOICE:-A} # A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU)


This and other ARG should just be the value assignment.

Why are they using var assignment?

Suggested change

ARG GPU_CHOICE=${GPU_CHOICE:-A} # A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU)

# A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU)

ARG GPU_CHOICE=A

You can override that default with --build-arg GPU_CHOICE=B, there is no need to try interpolate anything here like ENV.

polarathene · 2024-08-17T22:34:07Z

docker/Dockerfile

 ARG TORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX}"
 ARG BUILD_EXTENSIONS="${BUILD_EXTENSIONS:-}"
 ARG APP_UID="${APP_UID:-6972}"
 ARG APP_GID="${APP_GID:-6972}"

+WORKDIR /builder


What is the point of this directive? You don't use /builder in the next RUN and the next WORKDIR that follows changes the location again.

polarathene · 2024-08-17T22:37:19Z

docker/Dockerfile

 RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,rw \
    apt update && \
    apt install --no-install-recommends -y git vim build-essential python3-dev pip bash curl && \
    rm -rf /var/lib/apt/lists/*
 WORKDIR /home/app/
-RUN git clone https://github.com/oobabooga/text-generation-webui.git 
+RUN git clone https://github.com/oobabooga/text-generation-webui.git


A Dockerfile for a project it resides in should really just be sourcing the context of the repo already available.

Since this is already what the image has been doing, I'd defer changing this until a follow-up PR, but you should build from the repo with the repo root as the build context. The benefit is you can go to any tag/commit of the repo from that point onwards and build the image if needed. It's better for automation in CI too.

You've got a WORKDIR /home/app followed by the clone and then WORKDIR /home/app/text-generation-webui to change directories.

Suggested change

RUN git clone https://github.com/oobabooga/text-generation-webui.git

WORKDIR /home/app/text-generation-webui

RUN git clone https://github.com/oobabooga/text-generation-webui.git .

Remove the other two WORKDIR surrounding that suggestion, they're redundant.

polarathene · 2024-08-17T23:06:44Z

docker/Dockerfile

 # set umask to ensure group read / write at runtime
-CMD umask 0002 && export HOME=/home/app/text-generation-webui && ./start_linux.sh
+CMD umask 0002 && ./start_linux.sh $CLI_ARGS


umask 0002 seems unnecessary? Why is group write permission needed at runtime? it was introduced 9 months ago here, but no insights as to why.

CLI_ARGS was removed in March. Regardless, this is not the correct approach with Docker.

Suggested change

CMD umask 0002 && ./start_linux.sh $CLI_ARGS

ENTRYPOINT [ "/home/app/text-generation-webui/start_linux.sh" ]

Now CLI_ARGS would just be provided as the command (CMD):

docker run --rm -it image-name $CLI_ARGS

In compose.yaml the command field can set the CLI args to use, or you can use the Docker Compose ENV interpolation from a .env file to also use ${CLI_ARGS} as a value (kinda redundant though).

polarathene · 2024-08-17T23:08:01Z

docker/amd/docker-compose.yml

    env_file: .env
+    hostname: text-generation-webui


polarathene · 2024-08-17T23:14:08Z

docker/amd/docker-compose.yml

+        APP_GID: ${APP_GID:-6972}
+        APP_UID: ${APP_UID:-6972}


Are these actually used for anything? Nothing appears to reference them: https://github.com/search?q=repo%3Aoobabooga%2Ftext-generation-webui%20APP_UID&type=code

If they're only meant to change the runtime user, this should just be done with the standard user: uid:gid compose setting or docker run --user uid:gid CLI arg.

WolframRavenwolf and others added 8 commits February 9, 2024 19:26

Update .env.example

94cc0fc

Added PYTORCH_KERNEL_CACHE_PATH pointing at cache dir to fix "UserWarning: Specified kernel cache directory could not be created! This disables kernel caching."

Update .env.example

84a5287

Removed trailing slashes from cache paths for consistency

Delete docker/*/Dockerfile

0a07172

Update docker-compose.yml for unified Dockerfile

2034dc7

Update 09 - Docker.md

f42023b

Create symlinks and missing dirs

Update .env.example

ea65c64

Added APP_RUNTIME_UID to APP_RUNTIME_GID (so you can not only define the group, but also the user for access to mounted volumes)

Update .gitignore

47a8d8b

Added .dockerignore to .gitignore

oobabooga deleted the branch oobabooga:dev February 17, 2024 21:53

oobabooga closed this Feb 17, 2024

oobabooga reopened this Feb 17, 2024

polarathene suggested changes Aug 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed, improved, and unified Docker environment #5474

Fixed, improved, and unified Docker environment #5474

WolframRavenwolf commented Feb 9, 2024 •

edited

Loading

jtara1 commented Feb 19, 2024

oobabooga commented Jul 20, 2024

polarathene left a comment

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

polarathene Aug 17, 2024

	ARG GPU_CHOICE=${GPU_CHOICE:-A} # A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU)
	# A: NVIDIA, B: AMD, C: APPLE, D: INTEL, N: NONE (CPU)
	ARG GPU_CHOICE=A

	RUN git clone https://github.com/oobabooga/text-generation-webui.git
	WORKDIR /home/app/text-generation-webui
	RUN git clone https://github.com/oobabooga/text-generation-webui.git .

	CMD umask 0002 && ./start_linux.sh $CLI_ARGS
	ENTRYPOINT [ "/home/app/text-generation-webui/start_linux.sh" ]

Fixed, improved, and unified Docker environment #5474

Are you sure you want to change the base?

Fixed, improved, and unified Docker environment #5474

Conversation

WolframRavenwolf commented Feb 9, 2024 • edited Loading

Checklist:

Fixed and improved Docker environment

jtara1 commented Feb 19, 2024

oobabooga commented Jul 20, 2024

polarathene left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WolframRavenwolf commented Feb 9, 2024 •

edited

Loading