Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate ./dtox.sh image builds. #2233

Closed
jsirois opened this issue Aug 31, 2023 · 2 comments · Fixed by #2675 or #2679
Closed

Automate ./dtox.sh image builds. #2233

jsirois opened this issue Aug 31, 2023 · 2 comments · Fixed by #2675 or #2679
Assignees
Labels
tech-debt Issue that addresses technical debt.

Comments

@jsirois
Copy link
Member

jsirois commented Aug 31, 2023

Most of the infra is in place:

  • The ./dtox.sh script builds the base image only if needed, it just doesn't push the image.
  • The cache image used by CI was built and pushed by hand via tox -e build-cache-image -- --push.

The base image should be built in-line with CI (if needed), but the cache image need not be and is more expensive to build. That can probably be cronned at a period of ~1 week to start.

@jsirois jsirois added the tech-debt Issue that addresses technical debt. label Aug 31, 2023
@jsirois jsirois self-assigned this Feb 8, 2025
@jsirois
Copy link
Member Author

jsirois commented Feb 8, 2025

d365641 set up a cron job for this but it fails, hitting the GitHub 6 hour time limit.

I'm going to proceed instead with the idea of parallelizing the image build by only building some (or one) tox env per-shard, doing a docker export, then combing tarballs from the docker exports into one big one that is docker imported into 1 image in a fan-in step. A small experiment shows this works to consolidate a cache volume dir from multiple docker builds.

jsirois added a commit to jsirois/pex that referenced this issue Feb 13, 2025
Use this support to alter the CI job into a multi-sharded cache image
build job.

Fixes pex-tool#2233.
jsirois added a commit to jsirois/pex that referenced this issue Feb 14, 2025
@jsirois
Copy link
Member Author

jsirois commented Feb 14, 2025

OK, lots of thrash, but green as of manual run https://github.com/pex-tool/pex/actions/runs/13319142791. No individual job takes more than ~1:45 now (pypy310 IT), well under the 6 hour limit.

There was a remaining issue with the ./dtox.sh script fixed in #2679.

The cron job is already set up; so this should be all set.

jsirois added a commit that referenced this issue Feb 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-debt Issue that addresses technical debt.
Projects
None yet
1 participant