Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: remove set +e from swift package benchmark baseline compare #121

Merged
merged 1 commit into from
Oct 12, 2024

Conversation

kkebo
Copy link
Owner

@kkebo kkebo commented Oct 12, 2024

swift package benchmark baseline check may exit with status code 4 even if there are no performance regressions, but swift package benchmark baseline compare always returns status code 0 or 1.

@kkebo kkebo self-assigned this Oct 12, 2024
Copy link
Contributor

Code Metrics Report

main (48d5b3d) #121 (d548e16) +/-
Coverage 85.6% 85.5% -0.2%
Details
  |           | main (48d5b3d) | #121 (d548e16) |  +/-  |
  |-----------|----------------|----------------|-------|
- | Coverage  |          85.6% |          85.5% | -0.2% |
  |   Files   |             10 |             10 |     0 |
  |   Lines   |           1614 |           1607 |    -7 |
- |   Covered |           1382 |           1374 |    -8 |

Reported by octocov

Copy link
Contributor

Benchmark Summary

Sat Oct 12 01:46:50 UTC 2024

New baseline 'pull_request' is WITHIN the 'main' baseline thresholds.


Comparing results between 'main' and 'pull_request'

Host 'd22bcd5f5866' with 4 'x86_64' processors with 15 GB memory, running:
#26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024

MyBenchmark

lipsum metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 11 11 11 11 11 11 12 100
pull_request 11 11 11 11 11 12 13 100
Δ 0 0 0 0 0 1 1 0
Improvement % 0 0 0 0 0 -9 -8 0

lipsum-zh metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 1771 1780 1783 1790 1795 1834 1862 100
pull_request 1768 1776 1781 1787 1795 1833 1863 100
Δ -3 -4 -2 -3 0 -1 1 0
Improvement % 0 0 0 0 0 0 0 0

medium-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 41 42 42 42 42 48 49 100
pull_request 42 42 42 42 42 51 55 100
Δ 1 0 0 0 0 3 6 0
Improvement % -2 0 0 0 0 -6 -12 0

small-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4633 4657 4665 4674 4682 4735 4785 100
pull_request 4601 4628 4633 4645 4661 4768 5245 100
Δ -32 -29 -32 -29 -21 33 460 0
Improvement % 1 1 1 1 0 -1 -10 0

strong metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (μs) * p0 p25 p50 p75 p90 p99 p100 Samples
main 20 20 20 20 20 21 21 100
pull_request 20 20 20 20 21 21 23 100
Δ 0 0 0 0 1 0 2 0
Improvement % 0 0 0 0 -5 0 -10 0

tiny-fragment metrics

Time (wall clock): results within specified thresholds, fold down for details.

Time (wall clock) (ns) * p0 p25 p50 p75 p90 p99 p100 Samples
main 446 449 450 451 459 480 502 100
pull_request 454 455 456 457 464 482 512 100
Δ 8 6 6 6 5 2 10 0
Improvement % -2 -1 -1 -1 -1 0 -2 0

@kkebo kkebo merged commit 6929e6c into main Oct 12, 2024
1 of 2 checks passed
@kkebo kkebo deleted the remove-set-e-from-compare branch October 12, 2024 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant