[MRG] fix: relax test for poisson drive stats #978
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a trivial fix for stochastic, difficult-to-reproduce test failure. Rarely, and seemingly only on Python 3.8
Unit Test
runners, this test will fail due to the random Poisson mean interval exceeding theapprox
threshold. Here is an example case: https://github.com/asoplata/hnn-core/actions/runs/12952951618/job/36131478285#step:9:164 . I noticed the same thing happening on the first time I tried to run the tests for #971 ; however, since I told it to re-run the failed jobs, now I can't find any data on the previously-failed jobs. (It's possible that when you re-run failed jobs, maybe you can't access the old failed data using GitHub Actions?). As you can see in the above example, the test failed because the mean event interval was around 18.98, which is just outside of the threshold of 20 +/- 1.This may have been a pre-existing issue that sometimes produced random failures in the tests, but only rarely and stochastically. By expanding the tolerance for this test, this should become far less likely to cause test failures in the future.
To be clear, scientifically, we're saying that the mean of our realized Poisson rate is within 1.5 ([Hz] I believe) of our expected value, instead of 1 [Hz], for testing purposes. We could also tighten the tolerance gap back to something like 1.1 if anyone else is uncomfortable with this change (@ntolley ?)