v1.3.2 Performance Testing

[EnriqueL8]

Performance Runs

A big fix has gone into v1.3.2 to handle large numbers correctly and we have mode to using the decoder instead of unmarshall, just a blog I came across performance https://dev.to/jpoly1219/to-unmarshal-or-to-decode-json-processing-in-go-explained-5870 that shows a minimal impact in performance

v1.3.2 RC2

Started: 13/09/24
Duration: ~64 hours
Git commit: f0cdcff

Node Configuration
2 FireFly nodes on one virtual server (EC2 m4.xlarge)
Entire FireFly stack is local to the server (ie both blockchains, Postgres databases, etc)
Single geth node with 2 instances of ethconnect
Maximum time to confirm before considering failure = 1 minute

Summary


INFO[2024-09-16T19:08:58.591] Shutdown summary:
INFO[2024-09-16T19:08:58.593]  - Prometheus metric sent_mints_total        = 0.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric sent_mint_errors_total  = 0.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric mint_token_balance      = 0.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric received_events_total   = 12290484.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric incomplete_events_total = 0.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric delinquent_msgs_total    = 0.000000
INFO[2024-09-16T19:08:58.594]  - Prometheus metric actions_submitted_total = 6145402.000000
INFO[2024-09-16T19:08:58.594]  - Test duration: 79h25m44.554766275s
INFO[2024-09-16T19:08:58.594]  - Measured actions: 24580965
INFO[2024-09-16T19:08:58.594]  - Measured send TPS: 85.982304
INFO[2024-09-16T19:08:58.594]  - Measured throughput: 85.964095
INFO[2024-09-16T19:08:58.594]  - Measured send duration: min: 7.249484ms, max: 1.520039791s, avg: 145ms
INFO[2024-09-16T19:08:58.594]  - Measured event receiving duration: min: 2.847693759s, max: 25.117643682s, avg: 7.441s
INFO[2024-09-16T19:08:58.594]  - Measured total duration: min: 2.847693759s, max: 25.117643682s, avg: 7.441s

core-config.yml

log:
        level: debug
broadcast:
        batch:
                size: 200
                timeout: 1s
privatemessaging:
        batch:
                size: 200
                timeout: 1s
message:
        writer:
                count: 5
download:
        worker:
                count: 100
publicstorage:
                ipfs:
                        api:
                                requestTimeout: 2s
                        gateway:
                                requestTimeout: 2s

ethconnect.yml

cat: ethconnect.yml: No such file or directory

instances.yml

stackJSONPath: /home/ubuntu/.firefly/stacks/test/stack.json

wsConfig:
  wsPath: /ws
  readBufferSize: 16000
  writeBufferSize: 16000
  initialDelay: 250ms
  maximumDelay: 30s
  initialConnectAttempts: 5
  heartbeatInterval: 5s

instances:
  - name: long-run
    tests: [{"name": "msg_broadcast", "workers":50},{"name": "msg_private", "workers":50},{"name": "blob_broadcast", "workers":30},{"name": "blob_private", "workers":30}]
    length: 500h
    sender: 0
    recipient: 1
    messageOptions:
      longMessage: false
    tokenOptions:
      tokenType: fungible
    contractOptions:

FireFly git commit:

f0cdcffd000076cd817c0361213e5248c8986051

[EnriqueL8]

v1.3.2 RC3

Started: 01/10/24
Duration: ~27 hours - until out of memory
Git commit: c9bed10

Node Configuration
2 FireFly nodes on one virtual server (EC2 m4.xlarge)
Entire FireFly stack is local to the server (ie both blockchains, Postgres databases, etc)
Single geth node with 2 instances of ethconnect
Maximum time to confirm before considering failure = 1 minute

Summary

INFO[2024-10-02T11:21:57.557] Shutdown summary:
INFO[2024-10-02T11:21:57.558]  - Prometheus metric sent_mints_total        = 96939.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric sent_mint_errors_total  = 10.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric mint_token_balance      = 0.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric received_events_total   = 5733306.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric incomplete_events_total = 0.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric delinquent_msgs_total    = 0.000000
INFO[2024-10-02T11:21:57.558]  - Prometheus metric actions_submitted_total = 2769884.000000
INFO[2024-10-02T11:21:57.558]  - Test duration: 27h9m33.851204681s
INFO[2024-10-02T11:21:57.558]  - Measured actions: 11466612
INFO[2024-10-02T11:21:57.558]  - Measured send TPS: 117.348712
INFO[2024-10-02T11:21:57.558]  - Measured throughput: 117.276876
INFO[2024-10-02T11:21:57.558]  - Measured send duration: min: 5.495785ms, max: 1.736934973s, avg: 145ms
INFO[2024-10-02T11:21:57.558]  - Measured event receiving duration: min: 2.080458221s, max: 30.665500076s, avg: 6.7s
INFO[2024-10-02T11:21:57.558]  - Measured total duration: min: 2.080458221s, max: 30.665500076s, avg: 6.7s

core-config.yml

log:
        level: debug
broadcast:
        batch:
                size: 200
                timeout: 1s
privatemessaging:
        batch:
                size: 200
                timeout: 1s
message:
        writer:
                count: 5
download:
        worker:
                count: 100
publicstorage:
                ipfs:
                        api:
                                requestTimeout: 2s
                        gateway:
                                requestTimeout: 2s

ethconnect.yml

cat: ethconnect.yml: No such file or directory

instances.yml

stackJSONPath: /home/ubuntu/.firefly/stacks/rc3/stack.json

wsConfig:
  wsPath: /ws
  readBufferSize: 16000
  writeBufferSize: 16000
  initialDelay: 250ms
  maximumDelay: 30s
  initialConnectAttempts: 5
  heartbeatInterval: 5s

instances:
  - name: long-run
    tests: [{"name": "msg_broadcast", "workers":50},{"name": "msg_private", "workers":50},{"name": "blob_broadcast", "workers":30},{"name": "blob_private", "workers":30},{"name": "custom_ethereum_contract", "workers":20},{"name": "token_mint", "workers":10}]
    length: 500h
    sender: 0
    recipient: 1
    messageOptions:
      longMessage: false
    tokenOptions:
      tokenType: fungible
    contractOptions: {"address": "0xa92347b1ac51bf4c6f865687119cb130d0bd467b"}

FireFly git commit:

c9bed100202711e18d7065627746fdd8698e0b05

[EnriqueL8]

Performance looks all good! Marking as done