Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream test results without batched loads into server memory #73

Merged
merged 6 commits into from
Aug 1, 2023

Conversation

ayazhafiz
Copy link
Contributor

@ayazhafiz ayazhafiz commented Aug 1, 2023

Prior to this commit, an ABQ server would always load test results into memory before streaming them to the client. This could result in DOS of the server.

In this patch, we instead stream the results file to the client, but without loading the results into memory. At a given time, at most 8KB of a results file will be loaded in the server memory.

Note that the client still must load all results into memory, and can still suffer a DOS.

@github-actions
Copy link

github-actions bot commented Aug 1, 2023

Bigtest for e8cbf0c (run)

Benchmarks:

  • RSpec: 10.57% overhead
    • RSpec time: 17.79 seconds
    • ABQ time: 19.67 seconds
  • RSpec parallel, 10 runs: max 15.85% overhead
    • min 8.49% overhead
    • standard deviation: 2.11%
  • Jest: 5.75% overhead
    • Jest time: 20.352 seconds
    • ABQ time: 21.522 seconds

Fuzz result sizes:

  • PASSED

@ayazhafiz ayazhafiz merged commit 8567a8c into main Aug 1, 2023
17 checks passed
@ayazhafiz ayazhafiz deleted the copy-results-streaming branch August 1, 2023 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants