removes redundant vector allocations before calling sendmmsg::batch_send #4129
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
2 times, most recently
from
365214b to
43d93a0
Compare
behzadnouri
added
noCI
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
2 times, most recently
from
19acb5b to
bbb62dc
Compare
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
3 times, most recently
from
d73c5db to
0142799
Compare
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
from
0142799 to
8ed7047
Compare
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
2 times, most recently
from
6cedd30 to
1e221d9
Compare
alexpyattaev
reviewed
Feb 8, 2025
| @@ -1249,14 +1249,13 @@ impl ServeRepair { | |||
| } | |||
| } | |||
| if !pending_pongs.is_empty() { | |||
| match batch_send(repair_socket, &pending_pongs) { | |||
| let num_pkts = pending_pongs.len(); | |||
| let pending_pongs = pending_pongs.iter().map(|(bytes, addr)| (bytes, addr)); | |||
There was a problem hiding this comment.
this looks redundant... why map without any actual transform?
There was a problem hiding this comment.
The closure is converting &(S, T) to (&S, &T).
The code will not build without it.
There was a problem hiding this comment.
Oh I see. rust sometimes works in mysterious ways!
alexpyattaev
reviewed
Feb 8, 2025
alexpyattaev
reviewed
Feb 8, 2025
| @@ -362,7 +362,7 @@ fn recv_send( | |||
| let data = pkt.data(..)?; | |||
| socket_addr_space.check(&addr).then_some((data, addr)) | |||
| }); | |||
| batch_send(sock, &packets.collect::<Vec<_>>())?; | |||
| batch_send(sock, packets.collect::<Vec<_>>())?; | |||
There was a problem hiding this comment.
why are we still doing collect here if we can pass iterators now?
There was a problem hiding this comment.
We need an ExactSizeIterator. A filter_map is not an ExactSizeIterator because the number of items filtered out are not known.
alexpyattaev
reviewed
Feb 8, 2025
streamer::sendmmsg::batch_send only requires an ExactSizeIterator: https://github.com/anza-xyz/agave/blob/566bb9565/streamer/src/sendmmsg.rs#L203-L204 https://github.com/anza-xyz/agave/blob/566bb9565/streamer/src/sendmmsg.rs#L166-L175 Collecting an iterator into a vector before calling batch_send is unnecessary and only adds overhead. In particular multi_target_send used in retransmitting shreds can be use without doing an additional vector allocation: https://github.com/anza-xyz/agave/blob/566bb9565/streamer/src/sendmmsg.rs#L219
behzadnouri
force-pushed
the
multi-target-send-rm-vec-collect
branch
from
1e221d9 to
774f0d8
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
streamer::sendmmsg::batch_sendonly requires anExactSizeIterator:https://github.com/anza-xyz/agave/blob/82347779f/streamer/src/sendmmsg.rs#L169-L175
Collecting an iterator into a vector before calling
batch_sendis unnecessary and only adds overhead.In particular
multi_target_sendused in retransmitting shreds can be use without doing an additional vector allocation:https://github.com/anza-xyz/agave/blob/82347779f/streamer/src/sendmmsg.rs#L197
Summary of Changes
Removed redundant vector allocations before calling
streamer::sendmmsg::batch_send.