Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Double our Docker Hub rate limit #88

Merged
merged 1 commit into from
Oct 14, 2024

Conversation

tianon
Copy link
Member

@tianon tianon commented Oct 14, 2024

This increases our rate limit from 100/min up to 200/min (with an immediate burst of up to 200 and at most 200 at once). This should be generally safe since the code will already very aggressively drain the pool of available "tokens" the minute it sees a 429 (thus only doing a trickle of requests anyhow), and we auto-retry on 429, but only after a delay of a full second per request.

If all goes well, this should help mitigate our amd64 deploy jobs that have crept back up to a full hour runtime (without adding further parallelization complexity yet).

This increases our rate limit from 100/min up to 200/min (with an immediate burst of up to 200 and at most 200 at once).  This should be generally safe since the code will already very aggressively drain the pool of available "tokens" the minute it sees a 429 (thus only doing a trickle of requests anyhow), and we auto-retry on 429, but only after a delay of a full second per request.

If all goes well, this should help mitigate our amd64 deploy jobs that have crept back up to a full hour runtime (without adding further parallelization complexity yet).
@tianon tianon requested a review from yosifkit as a code owner October 14, 2024 23:39
@tianon
Copy link
Member Author

tianon commented Oct 14, 2024

#88! 😀

Doc and Marty from the Back to the Future movie, as Doc explains that "when this baby hits 88 miles per hour, you're going to see some serious shit"

@yosifkit yosifkit merged commit 0b7aef3 into docker-library:main Oct 14, 2024
1 check passed
@yosifkit yosifkit deleted the double-rate-limit branch October 14, 2024 23:59
@tianon
Copy link
Member Author

tianon commented Oct 15, 2024

If all goes well, this should help mitigate our amd64 deploy jobs that have crept back up to a full hour runtime (without adding further parallelization complexity yet).

Confirmed success -- before, deploy-amd64 was taking almost exactly 1h every hour, and now it's only taking ~29m 52s each time. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants