Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: More parallelization and caching #2124

Merged
merged 26 commits into from
Sep 18, 2024

Conversation

larseggert
Copy link
Collaborator

@larseggert larseggert commented Sep 17, 2024

  • Only use sccache on self-hosted runner. (GitHub storage limit is too small).
  • Break out fuzz and bench test compiles.
  • Only install and run llvm-cov on stable.
  • Add NSS_PREBUILT flag to neqo-crypto to avoid double-compile.

This PR roughly cuts the CI time in half.

Copy link

codecov bot commented Sep 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.35%. Comparing base (0e40d36) to head (b15f464).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2124   +/-   ##
=======================================
  Coverage   95.35%   95.35%           
=======================================
  Files         112      112           
  Lines       36334    36334           
=======================================
  Hits        34646    34646           
  Misses       1688     1688           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@larseggert larseggert changed the title ci: Try and use artifacts to cache prebuilt NSS ci: Cache prebuilt NSS Sep 17, 2024
Copy link

github-actions bot commented Sep 17, 2024

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

github-actions bot commented Sep 17, 2024

Benchmark results

coalesce_acked_from_zero 1+1 entries:
       time:   [99.495 ns 99.829 ns 100.17 ns]
Found 13 outliers among 100 measurements (13.00%)
  9 (9.00%) high mild
  4 (4.00%) high severe
coalesce_acked_from_zero 3+1 entries:
       time:   [117.28 ns 118.09 ns 119.29 ns]
Found 18 outliers among 100 measurements (18.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  15 (15.00%) high severe
coalesce_acked_from_zero 10+1 entries:
       time:   [116.66 ns 117.08 ns 117.61 ns]
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
  6 (6.00%) high severe
coalesce_acked_from_zero 1000+1 entries:
       time:   [97.323 ns 97.531 ns 97.774 ns]
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
RxStreamOrderer::inbound_frame():
       time:   [111.98 ms 112.03 ms 112.07 ms]
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) low mild
  4 (4.00%) high mild
transfer/pacing-false/varying-seeds:
       time:   [27.625 ms 28.726 ms 29.809 ms]
transfer/pacing-true/varying-seeds:
       time:   [35.175 ms 36.788 ms 38.473 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
transfer/pacing-false/same-seed:
       time:   [32.007 ms 32.743 ms 33.439 ms]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
transfer/pacing-true/same-seed:
       time:   [39.305 ms 41.714 ms 44.151 ms]
1-conn/1-100mb-resp (aka. Download)/client:
       time:   [113.59 ms 114.43 ms 115.64 ms]
       thrpt:  [864.74 MiB/s 873.93 MiB/s 880.37 MiB/s]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low severe
  6 (6.00%) low mild
  1 (1.00%) high severe
1-conn/10_000-parallel-1b-resp (aka. RPS)/client:
       time:   [313.52 ms 317.25 ms 320.99 ms]
       thrpt:  [31.153 Kelem/s 31.521 Kelem/s 31.896 Kelem/s]
1-conn/1-1b-resp (aka. HPS)/client:
       time:   [34.166 ms 34.381 ms 34.622 ms]
       thrpt:  [28.884  elem/s 29.086  elem/s 29.269  elem/s]
Found 15 outliers among 100 measurements (15.00%)
  6 (6.00%) low mild
  1 (1.00%) high mild
  8 (8.00%) high severe

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 198.0 ± 107.9 99.3 429.9 1.00
neqo msquic reno on 262.6 ± 68.1 205.8 429.7 1.00
neqo msquic reno 275.7 ± 92.3 214.0 439.5 1.00
neqo msquic cubic on 221.1 ± 12.9 202.6 237.1 1.00
neqo msquic cubic 300.6 ± 93.7 213.1 464.4 1.00
msquic neqo reno on 176.9 ± 98.9 92.0 365.0 1.00
msquic neqo reno 141.1 ± 85.5 81.7 325.2 1.00
msquic neqo cubic on 126.3 ± 72.0 82.1 324.8 1.00
msquic neqo cubic 151.1 ± 97.9 81.1 357.3 1.00
neqo neqo reno on 253.4 ± 141.1 137.4 594.7 1.00
neqo neqo reno 230.4 ± 125.5 141.5 586.4 1.00
neqo neqo cubic on 211.0 ± 93.4 133.4 496.5 1.00
neqo neqo cubic 189.0 ± 75.3 126.6 349.9 1.00

⬇️ Download logs

@larseggert larseggert marked this pull request as ready for review September 18, 2024 05:45
@larseggert larseggert changed the title ci: Cache prebuilt NSS ci: More parallelization and caching Sep 18, 2024
@larseggert larseggert merged commit b780e53 into mozilla:main Sep 18, 2024
59 checks passed
@larseggert larseggert deleted the ci-artifact-nss branch September 18, 2024 10:00
Copy link

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant