Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmarks for System program instructions #3567

Merged

Conversation

tao-stones
Copy link

Problem

Part of #3364, to benchmark each System instructions to determine their static CU consumption.

Summary of Changes

add benches for each instruction, aiming to run through their happy-path.

Fixes #

@tao-stones
Copy link
Author

results for 13 System instrucitons
     Running benches/system.rs (target/release/deps/system-fb7fac5a466a1157)
create_account          time:   [5.8636 µs 5.8711 µs 5.8782 µs]
                        change: [-0.4005% +0.2454% +0.8691%] (p = 0.45 > 0.05)
                        No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

create_account_with_seed
                        time:   [6.2564 µs 6.2668 µs 6.2783 µs]
                        change: [+3.5178% +3.9073% +4.2747%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

allocate                time:   [5.8454 µs 5.8568 µs 5.8689 µs]
                        change: [+3.2274% +3.7249% +4.2796%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe

allocate_with_seed      time:   [6.3466 µs 6.3611 µs 6.3780 µs]
                        change: [-0.5842% +0.3940% +1.3402%] (p = 0.43 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low severe
  7 (7.00%) high mild
  4 (4.00%) high severe

assign                  time:   [5.8576 µs 5.8736 µs 5.8929 µs]
                        change: [+1.0156% +1.4953% +1.9467%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

assign_with_seed        time:   [6.2664 µs 6.2866 µs 6.3107 µs]
                        change: [+2.9838% +3.2754% +3.5788%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

transfer                time:   [5.8182 µs 5.8319 µs 5.8449 µs]
                        change: [-0.2754% +0.1596% +0.5714%] (p = 0.46 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

transfer_with_seed      time:   [6.2091 µs 6.2299 µs 6.2561 µs]
                        change: [+1.1213% +1.4883% +1.8596%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe

initialize_nonce_account
                        time:   [14.623 µs 14.640 µs 14.657 µs]
                        change: [+0.5045% +0.7267% +0.9469%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

authorize_nonce_account time:   [6.1498 µs 6.1696 µs 6.1874 µs]
                        change: [-1.7916% -1.5224% -1.2104%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  10 (10.00%) high mild
  2 (2.00%) high severe

advance_nonce_account   time:   [14.753 µs 14.791 µs 14.831 µs]
                        change: [-0.1892% +0.2307% +0.6741%] (p = 0.28 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

upgrade_nonce_account   time:   [6.2506 µs 6.2608 µs 6.2720 µs]
                        change: [+0.2513% +0.7798% +1.3382%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  1 (1.00%) low severe
  8 (8.00%) high mild
  5 (5.00%) high severe

withdraw_nonce_account  time:   [14.496 µs 14.506 µs 14.517 µs]
                        change: [-1.8552% -1.4042% -0.9786%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe

@ksolana
Copy link

ksolana commented Nov 11, 2024

LGTM, variance in some tests are high (>3%) but we can figure that out as we go along. Thanks for setting up the infrastructure.

@tao-stones tao-stones merged commit 1c4a369 into anza-xyz:master Nov 14, 2024
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants