Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature suggestion: assert_all_same option to compare alternative implementations #435

Open
sabiwara opened this issue Aug 27, 2024 · 0 comments

Comments

@sabiwara
Copy link
Contributor

sabiwara commented Aug 27, 2024

Hi @PragTob, how are you doing?

What: an option like assert_all_same: true which would run all functions (for each input if using inputs) and ensure that results are equal for each input across functions.

Context: I've been mostly using Benchee to optimize pure functions in core, which means benchmarking alternative implementations. Failing early if one of my alternative implementation is not equivalent would improve the feedback loop, since I'm usually not writing tests yet during the benchmarking phase (but I often end up adding some dbg or manual checks).

I could write my own small wrapper, but since it might be a common use case I went for suggesting it in Benchee itself.

I'd be happy to send a PR if there is interest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant