-
-
Notifications
You must be signed in to change notification settings - Fork 294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate ./dtox.sh image builds. #2233
Comments
d365641 set up a cron job for this but it fails, hitting the GitHub 6 hour time limit. I'm going to proceed instead with the idea of parallelizing the image build by only building some (or one) tox env per-shard, doing a docker export, then combing tarballs from the docker exports into one big one that is docker imported into 1 image in a fan-in step. A small experiment shows this works to consolidate a cache volume dir from multiple docker builds. |
Use this support to alter the CI job into a multi-sharded cache image build job. Fixes pex-tool#2233.
Closes pex-tool#2233 for real.
OK, lots of thrash, but green as of manual run https://github.com/pex-tool/pex/actions/runs/13319142791. No individual job takes more than ~1:45 now (pypy310 IT), well under the 6 hour limit. There was a remaining issue with the The cron job is already set up; so this should be all set. |
Most of the infra is in place:
./dtox.sh
script builds the base image only if needed, it just doesn't push the image.tox -e build-cache-image -- --push
.The base image should be built in-line with CI (if needed), but the cache image need not be and is more expensive to build. That can probably be cronned at a period of ~1 week to start.
The text was updated successfully, but these errors were encountered: