[Into Pixi PR #4] Fix Dockerfiles #5
Conversation
jayqi
force-pushed
the
jyq-fix-dockerfiles
branch
from
2fdf926
to
c2290a5
Compare
| COPY apt.txt apt.txt | ||
| RUN apt-get update --fix-missing \ | ||
| && apt-get install -y apt-utils 2> /dev/null \ | ||
| && xargs -a apt.txt apt-get install -y \ | ||
| && apt-get clean \ | ||
| && rm -rf /var/lib/apt/lists/* /apt.txt | ||
|
|
||
| COPY --chown=$MAMBA_USER:$MAMBA_USER pixi.lock /tmp/pixi.lock | ||
| COPY --chown=$MAMBA_USER:$MAMBA_USER pixi.toml /tmp/pixi.toml |
There was a problem hiding this comment.
Not sure if you tried running your original, but pretty sure you'd get an error because the MAMBA_USER environment variable does not exist.
The mambaorg/micromamba base image is the thing that sets these environment variables and creates this user.
Since we're not using it anymore, then we don't have a non-root runtime user. We need to create it ourselves. The lines further up in the Dockerfile create the user.
There was a problem hiding this comment.
Huh I ran it without issue -- I wonder if it was somehow cached from when I built the image previously using the micromamba base. Either way, makes sense!
runtime/Dockerfile
Outdated
| COPY pixi.toml ./pixi.toml | ||
|
|
||
| RUN pixi install -e ${CPU_OR_GPU} --frozen \ | ||
| && rm -rf /home/${RUNTIME_USER}/.cache/ |
There was a problem hiding this comment.
pixi clean actually deletes the environment you install. This is the way to manually delete the cache directory.
There was a problem hiding this comment.
Ah got it I didn't realize "cleaning the environment" in the pixi docs meant deleting it.
It looks like we could also use the clean cache command, eg pixi clean cache --yes. Would that do the same thing?
There was a problem hiding this comment.
Oh, yeah, I think you're right. I didn't find that in --help yesterday.
| docker run \ | ||
| --mount type=bind,source="$(shell pwd)"/runtime,target=/tmp \ | ||
| --rm \ | ||
| pixi-lock:local |
There was a problem hiding this comment.
Does this line copy pixi.lock over from the docker container back to the host?
There was a problem hiding this comment.
--mount type=bind,source="$(shell pwd)"/runtime,target=/tmp \ mounts the local runtime/ directory into the container, so anything the container does happens directly to the host filesystem's runtime/.
|
@jayqi thank you this looks great and all runs without issue! A couple small question in the comments -- mostly for my understanding, no code changes needed |
* Use pixi for dependency management * Fix tensorflow * Move shared dependencies into a base feature * move pixi files * remove conda-lock files * dockerfile scrap work * update dockerfile * use pixi docker base image * work updating test_lockfile * update make update-lockfiles for pixi * separate dockerfile to update pixi.lock * update lockfile * update makefile * remove test_lockfile.py adapted for pixi * makefile logging * [Into Pixi PR #4] Fix Dockerfiles (#5) * Fix Dockerfiles * Fix permissions and directories and stuff * Update test command * Fix command ordering * Use clean cache command * Add maximize build space action * Add more root reserve space * Remove unwanted software directly * Add some diagnostics * Print out pixi info * Fix typo --------- Co-authored-by: Jay Qi <[email protected]> * Use a cuda base image * Hardcode python executable for test * Remove extra Python * pass CPU_OR_GPU to entrypoint.sh * log cpu or gpu in entrypoint * use jammy for locking * test pixi.lock update * hacky pixi dependency fix * revert pixi.lock * only log in smoke tests * fix mounting for lockfile generation * revert makefile changes for testing * fix comment * omit trailing slash --------- Co-authored-by: Jay Qi <[email protected]> Co-authored-by: Katie Wetstone <[email protected]>
This is a PR into the branch for #4 to fix some issues with the Dockerfiles.
See comments for explanations.