Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark Testing #5845

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

andrehrferreira
Copy link

Benchmark Testing Implementation for Various Node.js Frameworks
This pull request introduces a comprehensive benchmarking test that compares the performance of several popular Node.js frameworks. The benchmarks were designed to measure each framework’s:

Requests per second: The number of requests the framework can handle per second.
Transfer rate: The amount of data transferred per second.
Latency: The average time taken to respond to requests.
Latency Stdev: The standard deviation in the response time across all requests.
Max Latency: The maximum latency observed during the tests.

Frameworks Tested:

uWebSockets.js (uws)
Node.js HTTP (http)
Fastify
Koa
Restify
Hapi
Express

Results Summary:
The table below highlights the key metrics recorded during the benchmarking:

(index) Framework Requests/sec Transfer/sec Latency Total Requests Transfer Total Latency Stdev Latency Max
0 cluster_fastify 138457.36 23.24MB 15.58ms 1388733 233.09MB 58.10ms 842.11ms
1 cluster_http 137054.68 23.27MB 14.56ms 1376920 233.74MB 52.84ms 802.10ms
2 cluster_koa 122922.41 20.51MB 14.97ms 1234474 206.03MB 52.39ms 823.15ms
3 cluster_hapi 94445.64 19.91MB 23.66ms 949082 200.03MB 83.97ms 1.10s
4 cluster_restify 93398.96 16.48MB 22.78ms 937893 165.47MB 79.86ms 1.09s
5 cluster_express 43577.99 9.89MB 60.64ms 437738 99.36MB 190.66ms 2.00s
6 single_uws 37081.93 3.71MB 27.45ms 372355 37.29MB 3.06ms 76.40ms
7 cluster_uws 35629.39 3.57MB 28.57ms 357699 35.82MB 1.82ms 95.33ms
8 single_http 24159.04 4.10MB 42.09ms 242998 41.25MB 7.12ms 235.08ms
9 single_fastify 23794.91 3.99MB 43.18ms 239197 40.15MB 11.90ms 343.66ms
10 single_koa 20263.96 3.38MB 50.25ms 204371 34.11MB 17.65ms 294.61ms
11 single_hapi 14163.93 2.99MB 71.82ms 142853 30.11MB 16.06ms 410.62ms
12 single_restify 14036.58 2.48MB 72.46ms 141118 24.90MB 13.49ms 527.51ms
13 single_express 6830.47 1.55MB 147.89ms 68993 15.66MB 29.75ms 626.75ms

@UlisesGascon
Copy link
Member

As commented in expressjs/discussions#261 (comment). Maybe we want to expand our tests and compare against releases rather than other frameworks as Fastify already provides great comparison tool: https://fastify.dev/benchmarks/ (https://github.com/fastify/benchmarks/). cc: @expressjs/express-tc

@IamLizu
Copy link
Member

IamLizu commented Sep 3, 2024

As discussed, we're prioritizing comparisons between different Express releases over benchmarking against other projects. Therefore, I’m closing this issue in reference to expressjs/discussions#261.

But thanks a ton for bringing this up @andrehrferreira 👍

@IamLizu IamLizu closed this Sep 3, 2024
@andrehrferreira
Copy link
Author

@IamLizu Thank you for your attention =)

@wesleytodd
Copy link
Member

While I agree we first and foremost care about comparing to ensure we are improving, it is very helpful to have this kind of information. While I have not had a chance to review this PR, I just want to say that medium term we will care very much about this, so I am not sure if makes sense to close this PR. It is just that we cannot do any of this work right now until we get into v5 and maybe even v6 in the next year.

@IamLizu IamLizu reopened this Sep 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants