Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul CI strategy #142

Merged
merged 39 commits into from
Nov 25, 2024
Merged

Overhaul CI strategy #142

merged 39 commits into from
Nov 25, 2024

Conversation

LilithHafner
Copy link
Owner

Should close #139

@LilithHafner
Copy link
Owner Author

The testset "Long runtime budget doesn't throw right away" failed on a8dfc4c

@LilithHafner
Copy link
Owner Author

6ac0d78, as advertised, fixes this. As demonstrated by the CI results passing afterword and the testset tests for it as demonstrated by it failing before.

@LilithHafner
Copy link
Owner Author

Total usage over the last 6 runs:

1h 58m 43s
Update checkout and cache
1h 11m 11s
NFC
1h 22m 22s
Remove caching
1h 11m 17s
Restore caching
1h 17m 48s
NFC
1h 14m 24s

It seems that caching is not saving any time. I'll remove it. We can consider reinstating caching if there are new features or improved performance and we can demonstrate measurable savings.

@LilithHafner LilithHafner mentioned this pull request Nov 25, 2024
@LilithHafner
Copy link
Owner Author

Total usage:
1.2.2: 2h 35m 33s
main: 4h 1m 4s
PR: 1h 13m 32s (and much lower if tests fail)

Serial time (estimated):
1.2.2: 23 minutes
main: 42 minutes
PR: 19 minutes

Coverage now includes

  • macOS aarch64 on julia lts, 1, and pre
  • 32 bit (ubuntu @ Julia nightly, 1.0.0 and 1; and windows @ Julia 1)

Coverage now excludes

  • x86_64 via rosetta on macos on all versions of Julia except for 1
  • 64-bit windows on nightly and 1.0.0

RegressionTests now excludes

  • rosetta (Julia lts, 1, pre, nightly)
  • ubuntu (Julia pre and lts)

RegressionTest now includes

  • macOS aarch64 julia pre

@LilithHafner LilithHafner merged commit 87a7587 into main Nov 25, 2024
21 checks passed
@LilithHafner LilithHafner deleted the lh/CI-overhaull branch November 25, 2024 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better CI strategy to reduce CI time and increase coverage
1 participant