Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark compute-budget instructions #3614

Merged

Conversation

tao-stones
Copy link

Problem

Part of #3364, to benchmark each compute-budget instructions to determine their static CU consumption.

Summary of Changes

add benches for each instruction, aiming to run through their happy-path.

Fixes #

[dev-dependencies]
criterion = { workspace = true }
solana-compute-budget = { workspace = true }
solana-runtime-transaction = { workspace = true }
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev-dependent on solana-runtime-transaction for bencher to access process_compute_budget_instructions () to process instructions, caused circular dep. Because:

  1. solana-runtime-transaction dep on solana-builtins-default-costs crate to access MAYBE_BUILTIN_KEY static map;
  2. solana-builtins-default-costs dep on solana-compute-budget-program (this) crate, in order to get compute_budget::id()

Commonly we can solve this by creating a separate bench-compute-budget-program crate alone side this crate, but that'd break "structure" set up so far. Foreseeing similar situation might exist for other programs, maybe we adapt this change for all bench projects? (eg instead creating benches\ within target crate, creating bench-xyz-program crate next to it) Or perhaps there are more elegant solution?

@tao-stones tao-stones force-pushed the test-add-compute-budget-program-benches branch from 5b5b8a3 to 2e1711a Compare November 15, 2024 22:58
@@ -0,0 +1,24 @@
[package]
name = "solana-compute-budget-program-bench"
publish = false
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unpublished

@tao-stones
Copy link
Author

tao-stones commented Nov 15, 2024

./cargo nightly bench --manifest-path programs/compute-budget-bench/Cargo.toml
request_heap_limit      time:   [10.088 ns 10.103 ns 10.117 ns]
                        change: [+3.5586% +3.8022% +4.0640%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

set_compute_unit_limit  time:   [9.8263 ns 9.8382 ns 9.8512 ns]
                        change: [+2.3728% +2.5434% +2.7343%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe

set_compute_unit_price  time:   [9.9295 ns 9.9456 ns 9.9618 ns]
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe

set_loaded_accounts_data_size_limit
                        time:   [9.8628 ns 9.8735 ns 9.8863 ns]
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe

@tao-stones tao-stones merged commit be47703 into anza-xyz:master Nov 19, 2024
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants