Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: allow custom tags that are evaluated when an error happens #38

Open
goalaleo opened this issue Sep 12, 2024 · 0 comments

Comments

@goalaleo
Copy link

goalaleo commented Sep 12, 2024

Currently it's possible to configure the gem to include the :error tag in failed jobs. It's implemented in the server middlewere here

It's also possible to include custom_tags, but this is evaluated before the perform method is called.

I would need a sort of combination of these two features - I want to include custom tags when an error occurrs. My specific use case is that we have an SLI which compares the ratio between failed and total jobs. It looks something like this

      sli:
        events:
          errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue"}[{{.window}}])) > 0 or vector(0)
          totalQuery: sum(rate(sidekiq_jobs_executed_total{queue="my_queue"}[{{.window}}])) > 0 or vector(1)

However, there are some errors which are expected, and I'd like to exclude those based on a tag

errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue", ignore_error_in_sli=false}[{{.window}}])) > 0 or vector(0)

Currently the only way implement something like this would be to use the :error tag and include all the possible error class names.

errorQuery: sum(rate(sidekiq_jobs_failed_total{queue="my_queue", error!~="ErrorClass1|ErrorClass2|ErrorClass3"}[{{.window}}])) > 0 or vector(0)

This however is quite cumbersome if there are a lot of error classes, and I'd prefer not to define the errors in the manifest files as we'd like to use the list elsewhere in the code (single source of truth).

If this sounds like a reasonable feature then I can work on a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant