Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

image-automation-controller in OOMKilled crash loop #600

Open
rokos-angus opened this issue Nov 10, 2023 · 4 comments
Open

image-automation-controller in OOMKilled crash loop #600

rokos-angus opened this issue Nov 10, 2023 · 4 comments

Comments

@rokos-angus
Copy link

My image-automation-controller is constantly OOMKilled and gets stuck in a crash loop
Using image ghcr.io/fluxcd/image-automation-controller:v0.33.0

Resource limits
Limits: ││ cpu: 1 ││ memory: 1Gi

State: Running ││ Started: Fri, 10 Nov 2023 10:52:02 +0000 ││ Last State: Terminated ││ Reason: OOMKilled ││ Exit Code: 137 ││ Started: Fri, 10 Nov 2023 10:50:29 +0000 ││ Finished: Fri, 10 Nov 2023 10:51:46 +0000

Didn't see anything particularly useful in logs. We have around 15 ImageUpdateAutomation with a poll of 1 minute

@makkes
Copy link
Member

makkes commented Nov 10, 2023

Would you mind collecting a profile and sharing it here, please? Instructions here: https://fluxcd.io/flux/gitops-toolkit/debugging/#collecting-a-profile

@rokos-angus
Copy link
Author

Sure, github didn't like the extension but renamed it to txt. Didn't catch it in the crashloop but was running very high on cpu/mem at the time
heap.txt

@hiddeco
Copy link
Member

hiddeco commented Nov 10, 2023

Any chance one of the repositories has quite large files in it?

@rokos-angus
Copy link
Author

Just checked and the repository is about 140MB in total with all branches

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants