Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate alerts rules for missing metrics for each unit #177

Open
err404r opened this issue Aug 27, 2024 · 1 comment
Open

Generate alerts rules for missing metrics for each unit #177

err404r opened this issue Aug 27, 2024 · 1 comment

Comments

@err404r
Copy link

err404r commented Aug 27, 2024

Bug Description

For now COS can't detect the situation when there are no metrics are coming from grafana-agent. Ideally each registered grafana agent should generate one alert rule. And alert when metrics are missing

To Reproduce

Deploy grafana-agent, relate it to cos and break grafana-agent config.
In real life it could happened when tls_insecure_skip_verify=true is not set when it is required

Environment

grafana-agent revision 134

Relevant log output

No related logs

Additional context

No response

@simskij
Copy link
Member

simskij commented Aug 30, 2024

Some kind of up and absent type of rule needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants