Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase maximum startup time threshold (0.3 -> 0.4 s) #725

Merged
merged 1 commit into from
Aug 10, 2023

Conversation

osma
Copy link
Member

@osma osma commented Aug 10, 2023

We have a CI test for startup time, implemented in PR #706, designed to prevent situations where basic Annif CLI commands suddenly take a long time due to e.g. slow imports.

The threshold value was set to 0.300 seconds. But nowadays this test quite often fails because the startup takes a bit longer, e.g. 0.35 seconds. I don't think anything has changed with the imports, it's just that the CI environment runs a bit slower than it used to.

This PR increases the threshold value to 0.400 seconds, which should prevent spurious test failures but still ensure that startup times have not increased dramatically.

@osma osma added this to the 1.0 milestone Aug 10, 2023
@osma osma self-assigned this Aug 10, 2023
@sonarcloud
Copy link

sonarcloud bot commented Aug 10, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@osma
Copy link
Member Author

osma commented Aug 10, 2023

Ironically, the CI startup time test failed because startup took .432 seconds.

@codecov
Copy link

codecov bot commented Aug 10, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (e9031bb) 99.67% compared to head (2709b55) 99.67%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #725   +/-   ##
=======================================
  Coverage   99.67%   99.67%           
=======================================
  Files          89       89           
  Lines        6397     6397           
=======================================
  Hits         6376     6376           
  Misses         21       21           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@osma
Copy link
Member Author

osma commented Aug 10, 2023

...and then a re-run succeeded, when the startup time had decreased to .297 seconds. The test is clearly not very robust, but I think it's better to have a slightly higher threshold to reduce the amount of spurious test failures, so I'm merging this.

@osma osma merged commit 40cc2fd into main Aug 10, 2023
14 checks passed
@osma osma deleted the set-startup-time-threshold-0.4 branch August 11, 2023 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant