Throttling Logs to drop by group

[wildcardlinux]

I wanted to ask for a feature request to allow Fluent Bit to drop logs similar to this custom FluentD gem.

• https://github.com/rubrikinc/fluent-plugin-throttle?tab=readme-ov-file#fluent-plugin-throttle

A sentry plugin to throttle logs. Logs are grouped by a configurable key. When a group exceeds a configuration rate, logs are dropped for this group.

[patrick-stephens]

https://docs.fluentbit.io/manual/pipeline/filters/throttle - generally selection is done by matching in Fluent Bit.

[wildcardlinux]

Hello @patrick-stephens , the issue with this throttling is that it simply adjust the rate. What we are asking for is something that will drop messages similar to the FluentD gem linked above. Do you have anything that will drop messages as a throttle?

[patrick-stephens]

You can probably do it with a lua filter I imagine