Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Core team metrics #25

Merged
merged 1 commit into from
Mar 20, 2024

Conversation

fheinecke
Copy link

Hi folks. Internal tools would like to setup monitoring of some AWS metrics. I stumbled upon this repo while looking for a solution... really glad to see that somebody else already forked this 😀.

I've added a few metrics to monitor AMI quota limits, and ECR quota limits. Both of these have caused major problems when hit during security releases, delaying our releases. These resources have also cost us several hundreds of thousands of dollars when not cleaned up properly. If you guys don't mind owning this code, we'd like to add these metrics.

I have tested these changes locally against the release prod account, and they all work. The ECR image counter metric takes a long time to populate. We have 50k accounts and on my laptop it takes around a minute. We might want to increase the cache TTL. What do you think?

@evanfreed evanfreed requested a review from taraspos March 20, 2024 18:08
arn_parts = repository_arn.split(':')
service = arn_parts[2]
repository_name = arn_parts[5].removeprefix("repository/")
return get_paginated_results(session, service, "describe_images", "imageDetails", {'repositoryName': repository_name})
Copy link

@taraspos taraspos Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fheinecke did you try by chance list-images call, is it any faster?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested this. The decrease is negligible and within the margin of error for the tests I ran - around 100us per image. In addition the public ECR endpoint doesn't support list_images, so I'd need to add separate logic for both public and private ECR.

@taraspos taraspos merged commit a3406d0 into teleport Mar 20, 2024
4 checks passed
@taraspos taraspos deleted the fred/add-core-team-metrics-1 branch March 20, 2024 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants