Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Node Startup Config] Add ability to skip optimizer/sanitizer. #11502

Merged
merged 2 commits into from
Jan 12, 2024

Conversation

JoshLind
Copy link
Contributor

Description

This PR adds the ability to skip the node config optimizer and sanitizer on node startup. To achieve this, we add a new NodeStartupConfig which is used to determine if either step should be skipped. To disable sanitization and optimization, one would add this to their node config:

node_startup:
    skip_config_optimizer: true
    skip_config_sanitizer: true

The PR offers several commits:

  1. Rename the netbench.rs file to netbench_config.rs (for consistency -- all other files follow this format).
  2. Add the new NodeStartupConfig functionality and some small unit tests.

Test Plan

New and existing test infrastructure.

Copy link

trunk-io bot commented Dec 22, 2023

⏱️ 11h 48m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-unit-tests 3h 5m 🟩🟩🟩🟩
rust-smoke-tests 1h 58m 🟩🟩🟥🟩
windows-build 1h 15m 🟩🟩🟩🟩
forge-framework-upgrade-test / forge 1h 11m 🟥
execution-performance / single-node-performance 1h 2m 🟩🟩🟩
rust-lints 35m 🟩🟩🟩🟩
forge-e2e-test / forge 31m 🟥🟩
forge-compat-test / forge 31m 🟩🟩
rust-images / rust-all 27m 🟩🟥🟩
run-tests-main-branch 17m 🟩🟩🟩🟩
check 16m 🟩🟩🟩🟩
cli-e2e-tests / run-cli-tests 14m 🟩🟩
general-lints 11m 🟩🟩🟩🟩
check-dynamic-deps 7m 🟩🟩🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 2m 🟩🟩
semgrep/ci 1m 🟩🟩🟩🟩
execution-performance / file_change_determinator 49s 🟩🟩🟩
file_change_determinator 48s 🟩🟩🟩🟩
file_change_determinator 39s 🟩🟩🟩🟩
file_change_determinator 33s 🟩🟩🟩
execution-performance / parallel-execution-performance 25s 🟩🟩🟩
execution-performance / sequential-execution-performance 22s 🟩🟩🟩
permission-check 17s 🟩🟩🟩🟩
permission-check 13s 🟩🟩🟩🟩
permission-check 11s 🟩🟩🟩🟩
permission-check 10s 🟩🟩🟩🟩
permission-check 9s 🟩🟩🟩
determine-docker-build-metadata 6s 🟩🟩🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-images / rust-all 13m 11m +24%

settingsfeedbackdocs ⋅ learn more about trunk.io

@JoshLind JoshLind enabled auto-merge (rebase) January 11, 2024 22:36

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 59d711430767d1fc0f738d770fb0c05d3a6b5075

two traffics test: inner traffic : committed: 7834 txn/s, latency: 4938 ms, (p50: 4500 ms, p90: 6000 ms, p99: 14400 ms), latency samples: 3384680
two traffics test : committed: 100 txn/s, latency: 2323 ms, (p50: 2200 ms, p90: 2600 ms, p99: 7400 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.300, avg: 0.212", "QsPosToProposal: max: 0.166, avg: 0.143", "ConsensusProposalToOrdered: max: 0.642, avg: 0.574", "ConsensusOrderedToCommit: max: 0.505, avg: 0.477", "ConsensusProposalToCommit: max: 1.080, avg: 1.050"]
Max round gap was 1 [limit 4] at version 1660475. Max no progress secs was 4.985664 [limit 10] at version 1660475.
Test Ok

Copy link
Contributor

✅ Forge suite compat success on aptos-node-v1.8.3 ==> 59d711430767d1fc0f738d770fb0c05d3a6b5075

Compatibility test results for aptos-node-v1.8.3 ==> 59d711430767d1fc0f738d770fb0c05d3a6b5075 (PR)
1. Check liveness of validators at old version: aptos-node-v1.8.3
compatibility::simple-validator-upgrade::liveness-check : committed: 4277 txn/s, latency: 6595 ms, (p50: 6900 ms, p90: 8800 ms, p99: 13200 ms), latency samples: 183940
2. Upgrading first Validator to new version: 59d711430767d1fc0f738d770fb0c05d3a6b5075
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1781 txn/s, latency: 15861 ms, (p50: 19200 ms, p90: 21900 ms, p99: 22800 ms), latency samples: 92620
3. Upgrading rest of first batch to new version: 59d711430767d1fc0f738d770fb0c05d3a6b5075
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1323 txn/s, latency: 21558 ms, (p50: 21900 ms, p90: 29400 ms, p99: 30900 ms), latency samples: 67520
4. upgrading second batch to new version: 59d711430767d1fc0f738d770fb0c05d3a6b5075
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 2526 txn/s, latency: 9780 ms, (p50: 9900 ms, p90: 17500 ms, p99: 19700 ms), latency samples: 128860
5. check swarm health
Compatibility test for aptos-node-v1.8.3 ==> 59d711430767d1fc0f738d770fb0c05d3a6b5075 passed
Test Ok

Copy link
Contributor

❌ Forge suite framework_upgrade failure on aptos-node-v1.8.3 ==> 59d711430767d1fc0f738d770fb0c05d3a6b5075

Compatibility test results for aptos-node-v1.8.3 ==> 59d711430767d1fc0f738d770fb0c05d3a6b5075 (PR)
Upgrade the nodes to version: 59d711430767d1fc0f738d770fb0c05d3a6b5075
Test Failed: API error: Unknown error error sending request for url (http://aptos-node-3-validator.forge-framework-upgrade-pr-11502.svc:8080/v1/estimate_gas_price): error trying to connect: dns error: failed to lookup address information: Name or service not known

Stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: __libc_start_main
  15: <unknown>
Trailing Log Lines:
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: __libc_start_main
  15: <unknown>


Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at testsuite/forge/src/backend/k8s/swarm.rs:676:18:
called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-11502" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-11502\" not found", reason: "NotFound", code: 404 })

Caused by:
    namespaces "forge-framework-upgrade-pr-11502" not found: NotFound

Stack backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: __libc_start_main
  16: <unknown>
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:

@JoshLind JoshLind merged commit fc65fec into main Jan 12, 2024
40 of 44 checks passed
@JoshLind JoshLind deleted the config_san_skip branch January 12, 2024 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants