refactor connection management

[sharathjag]

Summary

1. Connection creation on every operation leaked resources leading to increased memory usage.
2. Similarly waiting for bucket to be ready on every operation introduced additional overhead and increased latency.

To mitigate both the issues, connection creation is done as a singleton and the bucket readiness is checked during the init phase. Further, connection to each buckets are managed by sync.Map to facilitate write once-read multiple pattern.

BUG=#14

Testing Done

at 100 VUs

INFO[0300] traffic_pattern: constant-vus. scenario: {"executor":"constant-vus","vus":"100","duration":"300s"}  source=console
INFO[0300] target_aggregate_qps: 1200000, num_workers: 30, qps_per_worker: 40000, max_measurable_latency: 2.5  source=console
INFO[0300] Thresholds: {"errorrate":[{"threshold":"rate<0.01","abortOnFail":true,"delayAbortEval":"27s"}],"calltime":[{"threshold":"p(99)<=1000","abortOnFail":true,"delayAbortEval":"27s"}]}  source=console

   ✓ calltime.............: avg=35.480258 min=17       med=37      max=369      p(90)=43      p(95)=46
     data_received........: 0 B    0 B/s
     data_sent............: 0 B    0 B/s
     errorcount...........: 0      0/s
   ✓ errorrate............: 0.00%  ✓ 0           ✗ 843480
     iteration_duration...: avg=35.54ms   min=314.91µs med=36.59ms max=368.78ms p(90)=42.69ms p(95)=46.38ms
     iterations...........: 843480 2809.282061/s
     vus..................: 100    min=100       max=100
     vus_max..............: 100    min=100       max=100


running (5m00.2s), 000/100 VUs, 843480 complete and 0 interrupted iterations
scenario_name ✓ [======================================] 100 VUs  5m0s

at 10000 VUs

INFO[0301] Thresholds: {"calltime":[{"threshold":"p(99)<=1000","abortOnFail":true,"delayAbortEval":"28s"}],"errorrate":[{"threshold":"rate<0.01","abortOnFail":true,"delayAbortEval":"28s"}]}  source=console

   ✓ calltime.............: avg=46.780789 min=17     med=32      max=647      p(90)=94       p(95)=136
     data_received........: 0 B      0 B/s
     data_sent............: 0 B      0 B/s
     errorcount...........: 25286    84.211911/s
   ✓ errorrate............: 0.10%    ✓ 25286        ✗ 23479321
     iteration_duration...: avg=107.24ms  min=29.5µs med=90.48ms max=842.87ms p(90)=170.28ms p(95)=222.3ms
     iterations...........: 23504607 78279.201031/s
     vus..................: 10000    min=0          max=10000
     vus_max..............: 10000    min=10000      max=10000


running (5m00.3s), 00000/10000 VUs, 23504607 complete and 0 interrupted iterations
scenario_name ✓ [======================================] 10000 VUs  5m0s

at 15000 VUs

INFO[0232] traffic_pattern: constant-vus. scenario: {"executor":"constant-vus","vus":"15000","duration":"300s"}  source=console
INFO[0232] target_aggregate_qps: 1200000, num_workers: 30, qps_per_worker: 40000, max_measurable_latency: 2.5  source=console
INFO[0232] Thresholds: {"calltime":[{"threshold":"p(99)<=1000","abortOnFail":true,"delayAbortEval":"30s"}],"errorrate":[{"threshold":"rate<0.01","abortOnFail":true,"delayAbortEval":"30s"}]}  source=console

   ✗ calltime.............: avg=137.000961 min=17      med=64       max=2500 p(90)=328      p(95)=408
     data_received........: 0 B      0 B/s
     data_sent............: 0 B      0 B/s
     errorcount...........: 3824     16.502807/s
   ✓ errorrate............: 0.02%    ✓ 3824         ✗ 15045661
     iteration_duration...: avg=207.02ms   min=38.12µs med=150.26ms max=2.5s p(90)=382.02ms p(95)=471.92ms
     iterations...........: 15049407 64947.035871/s
     vus..................: 15000    min=0          max=15000
     vus_max..............: 15000    min=14964      max=15000


running (3m51.7s), 00000/15000 VUs, 15047847 complete and 15258 interrupted iterations
scenario_name ✗ [============================>---------] 15000 VUs  3m51.2s/5m0s
ERRO[0235] thresholds on metrics 'calltime' were crossed; at least one has abortOnFail enabled, stopping test prematurely

Screenshots (if applicable)