Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimization(download): Consider using splice on linux to download snapshots #491

Open
3 tasks
InKryption opened this issue Jan 13, 2025 · 0 comments
Open
3 tasks
Labels
enhancement New feature or request

Comments

@InKryption
Copy link
Contributor

Currently in src/accountsdb/download.zig, we're downloading snapshots to disk using a cross-platform but naive approach, where our client reads the HTTP response body into memory, and then we write that data to the resulting file. This can result in a lot of context switching, since we're reading from kernelspace into userspace, and then writing from userspace back to kernelspace.

What we could instead do on linux is create a pipe, and splice the incoming response body through the pipe and then directly into the resulting file - this would avoid copying the data into userspace altogether.

Closing this issue will entail:

  1. Make this change in a pull request.
  2. Benchmark the performance difference.
  3. Merge the change if it's a performance gain, or close it if it's a loss or a wash.
@InKryption InKryption added the enhancement New feature or request label Jan 13, 2025
@0xNineteen 0xNineteen moved this to 📋 Backlog in Sig Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant