fluent bit output tail customize file name to separate logs for different days like log20250210.csv

[smsrezaei]

Is your feature request related to a problem? Please describe.
fluent bit output tail customize file name to separate logs for different days like log20250210.csv

Describe the solution you'd like
separate logs for different days like log20250210.csv

Describe alternatives you've considered
i have not seen alternative for this problem

Additional context
for my BI team i want serve log files daily
[INPUT]
Name tail
Path /app/log/*.csv
Read_from_Head true
Tag csvHelper
Parser csv_parser
Refresh_Interval 60

[FILTER]
Name lua
Match *
Script filter.lua
Call main

[OUTPUT]
Name file
Match *
Path /app/
file output.csv
Format template
Template {log}
Mkdir true

[patrick-stephens]

Are you asking that you want to customise the output filename in the file plugin (not the tail plugin)?

You can do that with the tag currently. I've done that previously, e.g. https://github.com/couchbase/couchbase-fluent-bit

[smsrezaei]

i need to modify output file name with output file plugin for each day ex. log_202500303.csv , i think do this job with output tag of lua , because in lua file i modify timestamp fields to date time and remove unnecessary fields .

[smsrezaei]

my issue like #2253 who open and unresolved

[smsrezaei]

i found answer

if you rewrite tag in lua

local new_record = record 
new_record["tag"] = convert_epoch_date(result[1]) 
new_record["log"] = t
return 2, timestamp, new_record

you can add filter to apply file name

[INPUT]
Name tail
Path /app/log/*.csv
Read_from_Head true
Tag csvHelper
Refresh_Interval 60

[FILTER]
Name lua
Match csvHelper
Script filter.lua
Call main

[FILTER]
Name rewrite_tag
Match csvHelper
Rule $tag ^.*$ $tag.csv false

[OUTPUT]
Name file
Match *
path /app/
Format template
Template {log}
Mkdir true