Benchmarking for api.chatie.io/v0/hosties/__TOKEN__

[huan]

Pressure Testing

We are running ab with the following args:

• -c 1000 1000 concurrency for multiple requests to make at a time
• -n 100000 100K of requests to perform
• http://chatieio.herokuapp.com/v0/hosties/puppet_wxwork_XXX this is a valid token

During the testing, we get the following results:

1. Failed Results: 0
2. Average Request Time: 4 seconds
3. Maximum Request Time: 13 seconds
4. Requests Per Second: 300

$ ab -n 100000 -c 1000  http://chatieio.herokuapp.com/v0/hosties/puppet_wxwork_XXX
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking chatieio.herokuapp.com (be patient)


Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        Cowboy
Server Hostname:        chatieio.herokuapp.com
Server Port:            80

Document Path:          /v0/hosties/puppet_wxwork_XXX
Document Length:        34 bytes

Concurrency Level:      1000
Time taken for tests:   331.766 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      27300000 bytes
HTML transferred:       3400000 bytes
Requests per second:    301.42 [#/sec] (mean)
Time per request:       3317.662 [ms] (mean)
Time per request:       3.318 [ms] (mean, across all concurrent requests)
Transfer rate:          80.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:      226  229   3.5    229    1231
Processing:   520 3066 963.5   2875   11938
Waiting:      499 3065 962.9   2874   11936
Total:        750 3295 963.5   3104   12169

Percentage of the requests served within a certain time (ms)
  50%   3104
  66%   3562
  75%   3816
  80%   4005
  90%   4556
  95%   5269
  98%   5800
  99%   6096
 100%  12169 (longest request)

Heroku Dyno Server Status

• Dyno Load: 1 ~ 2
• Node.js: Aggregate Time Spent in Garbage Collection: 13 s
• Node.js: Aggregate Garbage Collections: 15 k
• Node.js: Event Loop Latency: 40 ms
• Node.js: Event Loop Usage: 0.8 %

Conclusion

1. We can be safe when we have 1,000 concurrency requests to our API
2. The max capacity for API requesting is around 300 / s
3. It is sure that we will under Denial-of-Service (DoS) attack easily if there's anyone running an AB test on our API server with 100K concurrency.

@windmemory FYI