Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: run benchmarks (#62) #118

Merged
merged 1 commit into from
Oct 11, 2024
Merged

ci: run benchmarks (#62) #118

merged 1 commit into from
Oct 11, 2024

Conversation

kkebo
Copy link
Owner

@kkebo kkebo commented Oct 11, 2024

@kkebo kkebo self-assigned this Oct 11, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Oct 11, 2024

Benchmark Summary

Fri Oct 11 23:52:47 UTC 2024

===========================================
Threshold deviations for MyBenchmark:strong
===========================================
Time (wall clock) (ms, %) main pull_request Difference % Threshold %
p25 30 20 -32 5
p50 30 20 -32 5
p75 31 21 -32 5

New baseline 'pull_request' is BETTER than the 'main' baseline thresholds.


Comparing results between 'main' and 'pull_request'

Host '61c6ff3c7279' with 4 'x86_64' processors with 15 GB memory, running:
#26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024

MyBenchmark

lipsum metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 11 11 11 11 11 12 12 100
pull_request 11 11 11 11 11 11 11 100
Δ 0 0 0 0 0 -1 -1 0
Improvement % 0 0 0 0 0 8 8 0

lipsum-zh metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 1766 1775 1781 1784 1790 1828 1838 100
pull_request 1771 1781 1785 1804 1834 1884 1890 100
Δ 5 6 4 20 44 56 52 0
Improvement % 0 0 0 -1 -2 -3 -3 0

medium-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 42 42 42 42 42 42 43 100
pull_request 42 42 42 42 42 44 48 100
Δ 0 0 0 0 0 2 5 0
Improvement % 0 0 0 0 0 -5 -12 0

small-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4626 4645 4657 4665 4686 4739 4767 100
pull_request 4606 4624 4633 4645 4661 4784 4882 100
Δ -20 -21 -24 -20 -25 45 115 0
Improvement % 0 0 1 0 1 -1 -2 0

strong metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 30 30 30 31 31 31 44 100
pull_request 20 20 20 21 21 22 27 100
Δ -10 -10 -10 -10 -10 -9 -17 0
Improvement % 33 33 33 32 32 29 39 0

tiny-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 460 461 462 463 468 488 489 100
pull_request 454 456 456 458 466 483 486 100
Δ -6 -5 -6 -5 -2 -5 -3 0
Improvement % 1 1 1 1 0 1 1 0

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@kkebo kkebo changed the title ci: check performance regressions with CI ci: run benchmarks in CI (#62) Oct 11, 2024
@kkebo kkebo changed the title ci: run benchmarks in CI (#62) ci: run benchmarks (#62) Oct 11, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

Code Metrics Report

main (0096ab2) #118 (10246d5) +/-
Coverage 85.7% 85.4% -0.3%
Details
  |           | main (0096ab2) | #118 (10246d5) |  +/-  |
  |-----------|----------------|----------------|-------|
- | Coverage  |          85.7% |          85.4% | -0.3% |
  |   Files   |             10 |             10 |     0 |
  |   Lines   |           1597 |           1606 |    +9 |
+ |   Covered |           1370 |           1373 |    +3 |

Reported by octocov

@kkebo kkebo merged commit 48d5b3d into main Oct 11, 2024
3 checks passed
@kkebo kkebo deleted the issues/62 branch October 11, 2024 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Check performance regressions with CI
1 participant