Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add documentation to forge-stable test suite #14775

Merged
merged 2 commits into from
Jan 9, 2025
Merged

Add documentation to forge-stable test suite #14775

merged 2 commits into from
Jan 9, 2025
+95 −25

Conversation

igor-aptos
Copy link
Contributor

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@igor-aptos igor-aptos requested a review from a team as a code owner September 27, 2024 16:56
@trunk-io Trunk.io
Copy link

trunk-io bot commented Sep 27, 2024 •
edited
Loading

⏱️ 10m total CI duration on this PR
Job Cumulative Duration Recent Runs
check-dynamic-deps 3m 🟩 
forge-realistic-env-fairness-workload-sweep / forge 2m 🟥 
rust-cargo-deny 2m 🟩 
rust-move-tests 2m 🟩 
determine-test-metadata 2m 🟩 
semgrep/ci 27s 🟩 
general-lints 26s 🟩 
file_change_determinator 9s 🟩 
permission-check 7s 🟩 
permission-check 4s 🟩 
permission-check 3s 🟩 

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 3m 1m +117%
general-lints 26s 1m -71%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

igor-aptos
igor-aptos commented Sep 27, 2024
View reviewed changes
.github/workflows/forge-stable.yaml
# * chain progress - checking longest pause between the blocks (both in num failed runs, and in absolute time) across the run
# * catchup - that all nodes can go over the same version at the end of the load (i.e. catching if any individual validator got stuck)
# * no restarts - fails if any node has restarted within the test
# * system metrics - checks for CPU and RAM utilization during the test
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO - add network bandwidth metric check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disk iops?

@aluon aluon mentioned this pull request Oct 21, 2024
Merged
@github-actions GitHub Actions
Copy link
Contributor

This issue is stale because it has been open 45 days with no activity. Remove the stale label, comment or push a commit - otherwise this will be closed in 15 days.

@github-actions github-actions bot added the Stale label Nov 18, 2024
@github-actions github-actions bot closed this Dec 3, 2024
@igor-aptos igor-aptos removed the Stale label Dec 17, 2024
@igor-aptos igor-aptos reopened this Dec 17, 2024
@igor-aptos igor-aptos force-pushed the igor/forge_documentation branch 2 times, most recently from 0da1d72 to b3b4b09 Compare December 17, 2024 22:00
@igor-aptos igor-aptos added the CICD:build-images when this label is present github actions will start build+push rust images from the PR. label Dec 17, 2024
@igor-aptos igor-aptos force-pushed the igor/forge_documentation branch from b3b4b09 to f772a17 Compare December 21, 2024 00:43
@igor-aptos igor-aptos force-pushed the igor/forge_documentation branch from f772a17 to 9c5d9a1 Compare January 8, 2025 21:02
@igor-aptos igor-aptos enabled auto-merge (squash) January 8, 2025 21:02
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

✅ Forge suite compat success on 6593fb81261f25490ffddc2252a861c994234c2a ==> 9c5d9a11f3013ced3c47c669e709dec26425e535

Compatibility test results for 6593fb81261f25490ffddc2252a861c994234c2a ==> 9c5d9a11f3013ced3c47c669e709dec26425e535 (PR)
1. Check liveness of validators at old version: 6593fb81261f25490ffddc2252a861c994234c2a
compatibility::simple-validator-upgrade::liveness-check : committed: 17483.85 txn/s, latency: 1962.85 ms, (p50: 1800 ms, p70: 1900, p90: 2400 ms, p99: 6500 ms), latency samples: 576460
2. Upgrading first Validator to new version: 9c5d9a11f3013ced3c47c669e709dec26425e535
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 8061.87 txn/s, latency: 3720.00 ms, (p50: 4300 ms, p70: 4400, p90: 4500 ms, p99: 4700 ms), latency samples: 155820
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 8293.96 txn/s, latency: 4032.11 ms, (p50: 4400 ms, p70: 4500, p90: 4700 ms, p99: 5000 ms), latency samples: 275140
3. Upgrading rest of first batch to new version: 9c5d9a11f3013ced3c47c669e709dec26425e535
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7992.54 txn/s, latency: 3751.99 ms, (p50: 4300 ms, p70: 4500, p90: 4800 ms, p99: 5100 ms), latency samples: 148100
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 8036.94 txn/s, latency: 4181.79 ms, (p50: 4500 ms, p70: 4600, p90: 4700 ms, p99: 4900 ms), latency samples: 271800
4. upgrading second batch to new version: 9c5d9a11f3013ced3c47c669e709dec26425e535
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 6801.14 txn/s, latency: 4558.69 ms, (p50: 2700 ms, p70: 7400, p90: 8700 ms, p99: 8700 ms), latency samples: 118680
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 12049.42 txn/s, latency: 2717.04 ms, (p50: 2800 ms, p70: 3100, p90: 3300 ms, p99: 3600 ms), latency samples: 391100
5. check swarm health
Compatibility test for 6593fb81261f25490ffddc2252a861c994234c2a ==> 9c5d9a11f3013ced3c47c669e709dec26425e535 passed
Test Ok

@github-actions GitHub Actions

This comment has been minimized.

Sign in to view
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 9, 2025

✅ Forge suite realistic_env_max_load success on 9c5d9a11f3013ced3c47c669e709dec26425e535

two traffics test: inner traffic : committed: 14888.65 txn/s, latency: 2669.10 ms, (p50: 2700 ms, p70: 2700, p90: 2900 ms, p99: 3100 ms), latency samples: 5660980
two traffics test : committed: 99.99 txn/s, latency: 1346.09 ms, (p50: 1300 ms, p70: 1400, p90: 1500 ms, p99: 1700 ms), latency samples: 1780
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.564, avg: 1.492", "ConsensusProposalToOrdered: max: 0.289, avg: 0.287", "ConsensusOrderedToCommit: max: 0.309, avg: 0.301", "ConsensusProposalToCommit: max: 0.597, avg: 0.589"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.85s no progress at version 32038 (avg 0.19s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.88s no progress at version 6416998 (avg 0.71s) [limit 16].
Test Ok

@igor-aptos igor-aptos merged commit 0469e2d into main Jan 9, 2025
68 of 91 checks passed
@igor-aptos igor-aptos deleted the igor/forge_documentation branch January 9, 2025 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sherry-x sherry-x sherry-x approved these changes

@rustielin rustielin rustielin approved these changes

@perryjrandall perryjrandall Awaiting requested review from perryjrandall

@sitalkedia sitalkedia Awaiting requested review from sitalkedia

@zekun000 zekun000 Awaiting requested review from zekun000

Assignees
No one assigned
Labels
CICD:build-images when this label is present github actions will start build+push rust images from the PR.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@igor-aptos @rustielin @sherry-x