Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] main from mozilla:main #5

Open
wants to merge 430 commits into
base: main
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented May 7, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label May 7, 2024
larseggert and others added 29 commits September 13, 2024 12:48
And just compute it on the fly. It wasn't consistently updated anyway
and is only used in tests.
* ci: Unpin nss

Verbose, single-threaded build

* Fix

* Finalize
* ci: Fix Firefox build on Windows

* Exit

* Check

* WindowsSdkDir

* Finalize

* Also only PR-comment if we tried to build
* Replace xyber768 with mlkem768x25519

* rustfmt

---------

Co-authored-by: Lars Eggert <[email protected]>
…ns (#2111)

Bumps martenseemann/quic-network-simulator-endpoint from `f65a034` to `42d79cc`.

---
updated-dependencies:
- dependency-name: martenseemann/quic-network-simulator-endpoint
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(qpack): typo

* typo
If we pace, we might get the initial server flight before sending
sufficient 0-RTT data to pass the QNS check.

Broken out of #1998
Funtion is only used internally.

Broken out of #1998
* ci: Speed up NSS build with `sccache`

Maybe?

* Again

* No Windows

* Fix cmake

* Fixes

* Fixes

* echo

* Again

* exe

* No Windows

* No Windows

* Finalize

* Only set compiler env for the NSS build

---------

Signed-off-by: Lars Eggert <[email protected]>
* ci: Disable sccache on Windows again

Because for some reason today, the action doesn't install:
```
Error: Error: File was unable to be removed Error: EBUSY: resource busy or locked, rmdir 'C:\hostedtoolcache\windows\sccache\0.8.1\x64'
Error: File was unable to be removed Error: EBUSY: resource busy or locked, rmdir 'C:\hostedtoolcache\windows\sccache\0.8.1\x64'
```

* Check if it's installed already

* Fix
* fix: Don't encode large RTT guesses in tickets

Because under lossy conditions (e.g., QNS `handshakeloss` test), the
guess can be multiple times the actual RTT, which when encoded in the
resumption ticket will cause an extremely slow second handshake, often
causing the test to time out.

Broken out of #1998
Fixes #2088

* Fixes & tests

* Suggestion from @mxinden

* Fix
* `cargo machete`
* `cargo fmt`
* `cargo clippy`
* ci: Try and use artifacts to cache prebuilt NSS

* needs

* actions/cache

* Fix

* Fix

* no sccache

* sccache is killing our cache

* Set env

* Env

* NSS_PREBUILT

* Check if set

* Cache on self-hosted runner

* Fixes

* Fixes

* Fixes

* Run fuzz in parallel

* Invert

* fuzz-bench

* SCCACHE_INSTALLED && build -> check

* Fixes

* Fixes

* Don't update rustup

* Compile less
We don't track which packets are coalesced with others, so when we
detect a loss in one packet number space, we cannot resend any coalesced
packets in other packet number space. This PR tries to approximate this
behavior by scheduling un-ACKed Handshake and 0-RTT packets for RTX when
packets are lost in the Initial space.

Broken out of #1998
`add_datagram` takes a Quic datagram and adds it to the queue of datagrams to be
sent out. Previously it would take a reference (i.e. `&[u8]`) and would allocate
it into a new `Vec<u8>` before enqueuing. At the call-site the original
allocation (referenced by the `&[u8]`) would go out-of-scope and thus be
de-allocated. This is a wasted allocation for each Quic datagram to be send.

This commit has the call-site pass the owned `Vec<u8>` down right away.

Co-authored-by: Lars Eggert <[email protected]>
This caused some test failures, and @martinthomson discovered this
as the reason at the IETF 120 hackathon.

Broken out of #1998
So that we get issues for platform-specific code.

Also change `macos-14` -> `macos-latest` everywhere.
Nightly might have performance regressions.
The `neqo_transport::Connection::max_datagram_size` creates an `Encoder`, writes
a packet header and a packet number and determines how many bytes of the mtu are left.

https://github.com/mozilla/neqo/blob/28f60bd0ba3209ecba4102eec123859a3a8afd45/neqo-transport/src/connection/mod.rs#L3408-L3427

The `Encoder` only has to hold the packet header and the packet number. Yet it
is initialized with `Encoder::with_capacity(mtu)`.

https://github.com/mozilla/neqo/blob/28f60bd0ba3209ecba4102eec123859a3a8afd45/neqo-transport/src/connection/mod.rs#L3408

Note that `PacketBuilder::short` and `PacketBuilder::long` read the
`Encoder::capacity` through `PacketBuilder::infer_limit`. But
`PacketBuilder::infer_limit` falls back to `2048` if the capacity is below `64`,
which will be the case when using `Encoder::default()` instead of
`Encoder::with_capacity(mtu)`. `2048` should be plenty enough for the packet
header and the packet number.

https://github.com/mozilla/neqo/blob/28f60bd0ba3209ecba4102eec123859a3a8afd45/neqo-transport/src/packet/mod.rs#L152-L180

https://github.com/mozilla/neqo/blob/28f60bd0ba3209ecba4102eec123859a3a8afd45/neqo-transport/src/packet/mod.rs#L188-L225

https://github.com/mozilla/neqo/blob/28f60bd0ba3209ecba4102eec123859a3a8afd45/neqo-transport/src/packet/mod.rs#L135-L141

This commit prevents the wasted allocation by using `Encoder::default()` instead
of `Encoder::with_capacity(mtu)`.

Co-authored-by: Lars Eggert <[email protected]>
`Connection::add_packet_number -> PacketBuilder::pn -> Encoder::encode_uint`
panics when `pn_len` is `0`. These panics are seen in Firefox crash reports.

To be able to find the root cause of the panic, add additional metadata.

See #2132 for details.
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.7.0 to 6.8.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@5cd11c3...32945a3)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci(nss): don't set environment variables on BUILD_NSS 0

* Differentiate in system-nss, cached-nss, and nss-to-be-build
* fix: Add `confirmed` parameter to PTO calculation

Rather than having the caller determine for which space a PTO should be calculated for.

Broken out of #1998

I'm ambivalent if we want this change - thoughts?

* More from #1998

* More from #1998
This is simpler code overall.  It might even be faster.  It certainly
avoids an allocation.
* make process_output be able to return keep_alive timeout

* address comments

* address more comments
dependabot bot and others added 30 commits January 27, 2025 10:20
Bumps lukemathwalker/cargo-chef from `9d68a1b` to `8d7418b`.

---
updated-dependencies:
- dependency-name: lukemathwalker/cargo-chef
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.12.0 to 6.13.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@67a2d40...ca877d9)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.1.2 to 5.3.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@1e68e06...13ce06b)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Because otherwise we can't diagnose failed tests.
* chore: Factor out packet logging

Because `output_path` and `input_path` are getting long, and there is a lot of redundancy between the calls to `dump` and `qlog`.

* tos

* Minimize diff

* Minimize more

* More

* Less

* Fix len and tos

* clippy

* TODO

* Update neqo-transport/src/connection/mod.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

---------

Signed-off-by: Lars Eggert <[email protected]>
Co-authored-by: Martin Thomson <[email protected]>
* ci: Re-enable `Swatinem/rust-cache`

Now that we have `Cargo.lock` in the repository, we can re-enable the
`Swatinem/rust-cache` action. This will cache the `target` directory
between runs, which should speed up the CI process.

Also apply some other suggestions from https://corrode.dev/blog/tips-for-faster-ci-builds/

* Typo
…2403)

* ci: Create different Rust cache entries for different toolsets

Should unbreak CI.

* Again
Bumps lukemathwalker/cargo-chef from `8d7418b` to `ca7d16d`.

---
updated-dependencies:
- dependency-name: lukemathwalker/cargo-chef
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix: Enable MLKEM768X25519 by default

WIP

* Improve SNI slicing for large CIs

* WIP

* Finalize

* clippy

* Fixes

* MLKEM off in simulator

* Update neqo-transport/src/connection/mod.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Update neqo-transport/src/crypto.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Update neqo-transport/src/connection/mod.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Fixes

* SNI slicing revamp

* clippy

* Minimize diff

* Minimize

* clippy

* Comments

* Fix merge

---------

Signed-off-by: Lars Eggert <[email protected]>
Co-authored-by: Martin Thomson <[email protected]>
* ci: Use the `mold` linker on linux

Let's see if this speeds things up more.

* Fix

* Fix

* Handle own runner

* Fix

* Fix
* ci: Add `ubuntu-24.04-arm` to the `check` matrix

Fixes #2401

* Cache NSS for each arch
To hopefully reduce cache pressure.
* feat: In-place crypto

Only in-place encryption so far, and only for the main data path.

Fixes #2246 (eventually)

* aead_null

* WIP decrypt

* More

* fix(transport/packet): don't (mutably) borrow data multiple times

* Fixes

* Minimize

* Some suggestions from @martinthomson

* More

* More

* Fix `AeadNull`

* More suggestions from @martinthomson

* clippy

* fixme

* Update neqo-crypto/src/aead.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Update neqo-crypto/src/aead.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Update neqo-crypto/src/aead.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Update neqo-crypto/src/aead_null.rs

Co-authored-by: Martin Thomson <[email protected]>
Signed-off-by: Lars Eggert <[email protected]>

* Minimize diff

* Fix

---------

Signed-off-by: Lars Eggert <[email protected]>
Co-authored-by: Max Leonard Inden <[email protected]>
Co-authored-by: Martin Thomson <[email protected]>
* ci: Check CI workflows with `zizmor`

* ci: Enable `zizmor` CI workflow linting

And fix the issues.

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Add tags

* Fix

* debug

* Quotes

* Undo debug
* ci: Disable `arm64` docker image build

Because `gcc` on `arm64` segfaults. Yay.

Also don't run QNS on push to `main` to save some cycles.

* !cancelled()
It needs `security-events: write` to integrate with CodeQL.
* ci: Cache `hyperfine` results from `main`

So we can in a second PR compute deltas for PRs and display them.

* Ignore mv errors

* Is it mold?

* Nope

* Table improvements

* Don't wait for check during development of this

* Fix

* More

* Fix conditionals

* Fix

* Fix

* Fix

* Remove FIXME

* Fix
* ci: Fix `mold` on bencher

* Fix

* More
* ci: Debug QNS log failure

I hate GitHub CI

* Speed up

* More

* More

* More

* Again

* Minimize

* Finalize

* Fix highlight
* ci: Add delta to `main` to bench table

WIP

* Again

* Delta

* Fix

* More

* More

* More

* More

* Again

* Again

* More

* Again

* Again

* Again

* Again

* Again

* Again

* Again

* Minimize diff
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants