Skip to content

Commit

Permalink
Add crates.io Postmortem (July 21 2023) (#225)
Browse files Browse the repository at this point in the history
Signed-off-by: Rui Chen <[email protected]>
  • Loading branch information
chenrui333 authored Oct 9, 2023
1 parent 4fe0e3e commit dfa0c44
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@

[rust-lang](https://blog.rust-lang.org/inside-rust/2023/02/08/dns-outage-portmortem.html). On Wednesday, 2023-01-25 at 09:15 UTC, we deployed changes to the production infrastructure for crates.io. During the deployment, the DNS record for static.crates.io failed to resolve for an estimated time of 10-15 minutes. It was due to the fact that both certificates and DNS records were re-created during the downtime.

[rust-lang](https://blog.rust-lang.org/inside-rust/2023/07/21/crates-io-postmortem.html). On 2023-07-20 between 12:17 and 12:30 UTC all crate downloads from crates.io were broken due to a deployment that contained a bug in the download URL generation. During this time we had an average of 4.71K requests per second to crates.io, resulting in about 3.7M failed requests, including the retry attempts from cargo.

[Stack Overflow](https://web.archive.org/web/20201020103424/https://stackstatus.net/post/96025967369/outage-post-mortem-august-25th-2014). A bad firewall config blocked stackexchange/stackoverflow.

[Sentry](https://blog.sentry.io/2016/06/14/security-incident-june-12-2016). Wrong Amazon S3 settings on backups lead to data leak.
Expand Down

0 comments on commit dfa0c44

Please sign in to comment.