is /var/log/pods supported for kubernetes filter

[jvrahav]

Hi,

Im trying to use fluentbit as a sidecar container to one of my workload pods.
I have configured input for tag kube.* and Path refers to /var/log/pods
I have also configured Filter matching kube.* and Kube_Tag_Prefix as kube.var.log.pods.
With this configuration i see the logs are getting streamed however the kubernetes metadata is not being added to the logs.

I suspect that the kubernetes metadata filter does not work well with /var/log/pods

can someone please confirm if the behaviour is as expected meaning kubernetes filter does not support /var/log/pods

` [INPUT]
Name tail
Tag kube.*
#Path /var/log/pods/external-dnsexternal-dns-ns_external-dns.log
Path /var/log/pods//external-dns//*.log
Parser docker
#DB /var/log/vmware/fluentbit/flb_kube.db
Buffer_Max_Size 12MBb
Mem_Buf_Limit 32MB
Skip_Long_Lines On
Refresh_Interval 10

[FILTER]
    Name                kubernetes
    Match               kube.*
    Kube_URL            https://kubernetes.default.svc.cluster.local
    Kube_Tag_Prefix     kube.var.log.pods.
    tls.verify          Off
    K8S-Logging.Parser  On
    K8S-Logging.Exclude Off
    Annotations         Off
    Labels              On`

[xavierivadulla]

To use /var/log/pods I generated my own parser.

Basically on INPUT I added the new log path, on PARSER I declared the new parser and on FILTER I use the new filter.

[INPUT]
    Name tail
    Tag kube.*
    multiline.parser cri, docker
    Path /var/log/pods/*/*/*log

[PARSER]
    Name    kube-pod-log
    Format  regex
    Reserve_Data true
    Regex   (?<namespace_name>[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:_[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<pod_name>[^_]+)_(?<docker_id>.*)\.(?<container_name>.*)\..*\.log$

[FILTER]
    Name kubernetes
    Match kube.*
 
    Kube_Tag_Prefix kube.var.log.pods.
    Regex_Parser kube-pod-log

Hope this can help.

[jvrahav]

Thanks...we ended up doing the same