Skip to content

Commit

Permalink
fail on output
Browse files Browse the repository at this point in the history
  • Loading branch information
DawidNiezgodka committed Oct 18, 2023
1 parent 7a65872 commit fcc88c2
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 6 deletions.
10 changes: 10 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ jobs:

- name: evaluate benchmark
uses: ./
id: eval
with:
name: "Benchmark2"
bench_to_compare: "Benchmark"
Expand All @@ -50,6 +51,15 @@ jobs:
git config --local user.name "github-actions[bot]"
git commit -a -m "Add changes"
git push origin bench_data
- name: Check output and fail if needed
if: steps.eval.outputs.failed == 'true'
run: |
echo "Failing because output is some_value_to_fail_on"
exit 1
# - name: Push changes
# uses: ad-m/github-push-action@master
# with:
Expand Down
6 changes: 6 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,12 @@ inputs:
required: false
default: "any"

outputs:
should_fail:
description: |
Indicates whether the benchmark should fail or not.
The possible values are true and false.
runs:
using: node20
Expand Down
6 changes: 3 additions & 3 deletions dist/index.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions src/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,15 @@ async function run() {
config.fileWithBenchData
)
}

core.setOutput('should_fail', 'false')
if (config.failingCondition === 'any') {
if (anyFailed(resultArray)) {
core.setFailed('Some benchmarks are worse than the reference')
core.setOutput('should_fail', 'true')
}
}
if (config.failingCondition === 'all') {
if (allFailed(resultArray)) {
core.setFailed('All benchmarks are worse than the reference')
core.setOutput('should_fail', 'true')
}
}
} catch (error) {
Expand Down

1 comment on commit fcc88c2

@DawidNiezgodka
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark2

Benchmark information

Current Benchmark Previous Benchmark
Execution time: 138 m Execution time: 138 m
Parametrization: Parametrization:
- ack: 1 - ack: 1
- read-in-ram: true - read-in-ram: true
Other Info: kafka, v2.3 Other Info: kafka, v2.3

Results

Metric name Current: fcc88c2 Previous: 76bcc05 Condition Result
latency 200 ms 100 ms smaller 🔴
metric2 200 op/s 100 op/s bigger 🟢
metric3 200 % 100 % range 🔴

Please sign in to comment.