Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail asv workflows when performance deteriorates #252

Open
camposandro opened this issue Aug 9, 2023 · 0 comments
Open

Fail asv workflows when performance deteriorates #252

camposandro opened this issue Aug 9, 2023 · 0 comments
Labels
enhancement New feature or request new option Add option for existing practice, or new practice

Comments

@camposandro
Copy link
Contributor

camposandro commented Aug 9, 2023

Fail the ASV workflows if at least one of benchmarks deteriorates above a certain threshold (e.g. 150% / 200%).

asv-main.yml:

  • Execute asv run without || true, allowing the command to exit with a non-zero return code
  • Submit benchmarks and deploy dashboard on gh-pages, if the previous command succeeded or not

asv-pr.yml:

  • Execute asv continuous without || true, allowing the command to exit with a non-zero return code
  • Output the performance comment on the PR if the comparison was successful or not

asv-nightly.yml

  • Execute asv continuous without || true, allowing the command to exit with a non-zero return code
  • Write the commit hash to cache, if the previous if the comparison was successful or not

More information about "on failure" executions here.

@camposandro camposandro added enhancement New feature or request new option Add option for existing practice, or new practice labels Aug 9, 2023
@camposandro camposandro changed the title Allow failure of ASV workflow when performance deteriorates Fail asv workflows when performance deteriorates Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request new option Add option for existing practice, or new practice
Projects
None yet
Development

No branches or pull requests

1 participant