Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test suite is non-deterministic #411

Open
2 tasks
bsweger opened this issue Jan 26, 2024 · 0 comments
Open
2 tasks

Test suite is non-deterministic #411

bsweger opened this issue Jan 26, 2024 · 0 comments

Comments

@bsweger
Copy link
Collaborator

bsweger commented Jan 26, 2024

When the test suite runs locally and via CI, the results can vary between runs, even when the inputs haven't changed. The culprit is preprocess_jhu files has expected combinations of location, week

Here's an example of a test failure (running on macos-latest) followed by success on a re-run

  1. Attempt one - failure
  2. Attempt two - successful re-run

There's an open question of whether or not covidHubUtils needs to keep the data pre-processing functions for target data that is no longer being updated.

If we're keeping these functions and their associated tests, we should fix the cause of this erratic behavior. We might also want to refactor ``preprocess_jhu files has expected combinations of location, week into a true unit test, rather than calling out to external dependencies like `covidData` (the test suite takes me 30 minutes to run locally).

Definition of done:

  • re-running covidHubUtils test suite produces consistent results
  • covidHubUtils unit tests avoid external calls
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant