bench(udp): run GSO, GRO and recvmmsg permutations #2010

mxinden · 2024-10-09T18:35:04Z

Benchmark all permutations of GSO, GRO and recvmmsg.
Add support for all platforms.
Add test-run (i.e. cargo test --benches, single execution in debug mode) to CI.

mxinden · 2024-10-10T07:27:34Z

quinn-udp/Cargo.toml

@@ -30,8 +30,13 @@ once_cell = { workspace = true }
 windows-sys = { workspace = true }

 [dev-dependencies]
-criterion = "0.5"
+criterion = { version = "0.5", default-features = false, features = ["async_tokio"] }
+tokio = { workspace = true, features = ["rt", "rt-multi-thread", "net"] }


On Linux blocking recvmmsg calls will wait for all iovecs to be filled.

A blocking recvmmsg() call blocks until vlen messages have been
received or until the timeout expires. A nonblocking call reads
as many messages as are available (up to the limit specified by
vlen) and returns immediately.

https://man7.org/linux/man-pages/man2/recvmmsg.2.html

This is e.g. problematic when combined with a GSO send with less segments than iovecs in its recvmmsg counterpart.

Thus this pull request changes the benchmark to use non-blocking instead of blocking recvmmsg. To do so, it uses tokio.

See also discussion in #1993 (comment).

Suggest you keep a separate commit that makes the change from sync to async.

mxinden · 2024-10-10T07:28:27Z

quinn-udp/benches/throughput.rs

+        #[cfg(any(target_os = "linux", target_os = "windows"))]
+        true,


Other platforms don't support GSO.

Needs to add apple platforms once #1993 merges.

mxinden · 2024-10-10T07:30:00Z

quinn-udp/benches/throughput.rs

+const MAX_IP_UDP_HEADER_SIZE: usize = 48;
+const MAX_DATAGRAM_SIZE: usize = u16::MAX as usize - MAX_IP_UDP_HEADER_SIZE;


Linux allows for up to 64KiB per GSO and GRO call. That includes IP and UDP header.

mxinden · 2024-10-10T07:31:24Z

quinn-udp/benches/throughput.rs

+                            // recv.readable() can lead to false positives. Try again.
+                            Err(e) if e.kind() == ErrorKind::WouldBlock => continue,
+                            e => e.unwrap(),


UdpSocket::readable is allowed to return false-positives.

The function may complete without the socket being readable. This is a false-positive and attempting a try_recv() will return with io::ErrorKind::WouldBlock.

https://docs.rs/tokio/latest/tokio/net/struct.UdpSocket.html#method.readable

mxinden · 2024-10-10T07:33:27Z

.github/workflows/rust.yml

      - run: cargo test
      - run: cargo test --manifest-path fuzz/Cargo.toml
        if: ${{ matrix.rust }} == "stable"
+      - run: cargo test --benches


Note that I did not simply extend the existing cargo test call, but instead added an additional cargo test --benches.

cargo test --benches executes some but not all unit tests in addition to all benchmark tests. See also rust-lang/cargo#6454.

mxinden · 2024-10-10T07:35:33Z

@larseggert this pull request should resolve all issues raised in #1993 (review). Can you take a look?

Targeting main directly given that changes are not tied to #1993.

mxinden · 2024-10-10T14:10:22Z

@djc @Ralith ready for a full review. Let me know if you find this pull request helpful.

Will unblock #1993.

djc

I think this is probably a good change but would like to see more organization in terms of separate commits -- here's some early feedback.

djc · 2024-10-10T20:21:41Z

quinn-udp/Cargo.toml

@@ -30,8 +30,13 @@ once_cell = { workspace = true }
 windows-sys = { workspace = true }

 [dev-dependencies]
-criterion = "0.5"
+criterion = { version = "0.5", default-features = false, features = ["async_tokio"] }
+tokio = { workspace = true, features = ["rt", "rt-multi-thread", "net"] }


Suggest you keep a separate commit that makes the change from sync to async.

djc · 2024-10-10T20:22:05Z

quinn-udp/benches/throughput.rs

-use std::cmp::min;
-use std::net::{Ipv4Addr, Ipv6Addr};
-use std::{io::IoSliceMut, net::UdpSocket, slice};
+use quinn_udp::{RecvMeta, Transmit, UdpSocketState, BATCH_SIZE};


Nit: would be good to do StdExternalCrate import blocks here.

djc · 2024-10-10T20:22:14Z

quinn-udp/benches/throughput.rs

+use tokio::io::Interest;
+use tokio::runtime::Runtime;
+
+const MAX_IP_UDP_HEADER_SIZE: usize = 48;


Nit: constants should go near the bottom.

djc · 2024-10-10T20:22:43Z

quinn-udp/benches/throughput.rs

+    io::{ErrorKind, IoSliceMut},
+    net::{Ipv4Addr, Ipv6Addr, UdpSocket},
+};
+use tokio::io::Interest;


Nit: for consistency, there should be one import statement for all tokio imports.

djc · 2024-10-10T20:23:05Z

quinn-udp/benches/throughput.rs

                }
            })
        });
    }
 }

+fn new_socket(rt: &mut Runtime) -> (tokio::net::UdpSocket, UdpSocketState) {


Please keep the extraction of this function in a separate commit.

mxinden force-pushed the bench-throughput-gso-gro-recvmmsg branch 5 times, most recently from 62dd45e to 1406041 Compare October 10, 2024 07:16

mxinden mentioned this pull request Oct 10, 2024

bench(udp): support all GSO and recvmmsg permutations across platforms larseggert/quinn#1

Closed

bench(udp): run GSO, GRO and recvmmsg permutations

bc8825b

mxinden force-pushed the bench-throughput-gso-gro-recvmmsg branch from 1406041 to bc8825b Compare October 10, 2024 07:19

mxinden commented Oct 10, 2024

View reviewed changes

mxinden marked this pull request as ready for review October 10, 2024 07:34

mxinden mentioned this pull request Oct 10, 2024

feat: Faster UDP/IO on Apple platforms #1993

Open

larseggert approved these changes Oct 10, 2024

View reviewed changes

mxinden mentioned this pull request Oct 10, 2024

"unused manifest key" warning occurs upon cargo check #2004

Open

djc reviewed Oct 10, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bench(udp): run GSO, GRO and recvmmsg permutations #2010

bench(udp): run GSO, GRO and recvmmsg permutations #2010

mxinden commented Oct 9, 2024 •

edited

Loading

mxinden Oct 10, 2024

djc Oct 10, 2024

mxinden Oct 10, 2024

mxinden Oct 10, 2024

mxinden Oct 10, 2024

mxinden Oct 10, 2024

mxinden commented Oct 10, 2024

mxinden commented Oct 10, 2024

djc left a comment

djc Oct 10, 2024

djc Oct 10, 2024

djc Oct 10, 2024

djc Oct 10, 2024

djc Oct 10, 2024

		#[cfg(any(target_os = "linux", target_os = "windows"))]
		true,

		const MAX_IP_UDP_HEADER_SIZE: usize = 48;
		const MAX_DATAGRAM_SIZE: usize = u16::MAX as usize - MAX_IP_UDP_HEADER_SIZE;

bench(udp): run GSO, GRO and recvmmsg permutations #2010

Are you sure you want to change the base?

bench(udp): run GSO, GRO and recvmmsg permutations #2010

Conversation

mxinden commented Oct 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mxinden commented Oct 10, 2024

mxinden commented Oct 10, 2024

djc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mxinden commented Oct 9, 2024 •

edited

Loading