grafana-agent consumes a substantial amount of CPU when log folders have more than 100K files.

[drencrom]

Enhancement Proposal

When grafana-agent is scraping log folders with hundreds of thousands of files it creates thousands of threads and consumes a lot of CPU time. It would be desirable to be able to customize the operator so that some log folders can be excluded (using the __path_exclude__ directive for example).
This problem was observed in particular with the /var/log/libvirt/qemu/ folder. There is a related enhancement proposal in loki (canonical/loki-k8s-operator#439) regarding the huge cardinality of the scraped data that might be tackled together with this one.