Skip to content

Commit

Permalink
out_kafka: add support for dynamic/static headers
Browse files Browse the repository at this point in the history
Signed-off-by: difrin <[email protected]>
  • Loading branch information
DIFRIN committed May 29, 2024
1 parent 13c922a commit 3dec0b6
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions pipeline/outputs/kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Kafka output plugin allows to ingest your records into an [Apache Kafka](https:/
| dynamic\_topic | adds unknown topics \(found in Topic\_Key\) to Topics. So in Topics only a default topic needs to be configured | Off |
| queue\_full\_retries | Fluent Bit queues data into rdkafka library, if for some reason the underlying library cannot flush the records the queue might fills up blocking new addition of records. The `queue_full_retries` option set the number of local retries to enqueue the data. The default value is 10 times, the interval between each retry is 1 second. Setting the `queue_full_retries` value to `0` set's an unlimited number of retries. | 10 |
| rdkafka.{property} | `{property}` can be any [librdkafka properties](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) | |
| Header | Add a HTTP header key/value pair. Multiple headers can be. If static header than use Header key value, if the value is part of the message than use Header key <value> | |

> Setting `rdkafka.log.connection.close` to `false` and `rdkafka.request.required.acks` to 1 are examples of recommended settings of librdfkafka properties.
Expand Down Expand Up @@ -45,6 +46,7 @@ In your main configuration file append the following _Input_ & _Output_ sections
Match *
Brokers 192.168.1.3:9092
Topics test
Header test test
```

### Avro Support
Expand Down

0 comments on commit 3dec0b6

Please sign in to comment.