[REQUEST]: Application logs add "Elastic Agent" option

[SylvainJuge]

Description

With the new section on the Application Logs, we document the ingestion strategies that mostly rely on the semi-deprecated Filebeat.

We need to provide documentation how to achieve the same with a Fleet integration, maybe with the "Custom Logs" + some configuration for now.

Also, we could take the opportunity to re-structure a bit this guide by relying on the log format.

• Plain-text logs
  • Changes required in app/config
  • Filebeat/Fleet configuration to ingest those logs (both would require an ingest pipeline).
• ECS formatted logs
  • with ECS Logging library
  • with ECS Reformatting (and thus APM agent)
  • Filebeat/Fleet configuration to ingest those logs
• Log sending

Or, alternatively, structure the guide by relying on the ingestion tool as top-level:

• Elastic Agent
• Filebeat
• APM Agent

Resources

current Application Logs doc: https://www.elastic.co/guide/en/observability/current/application-logs.html

Collaboration

TBD. The docs and product team will work together to determine the best path forward.

Point of contact.

Main contact: @SylvainJuge

Stakeholders:

[SylvainJuge]

Initialy mentioned here : elastic/apm#765 (comment)

[felixbarny]

I hope we'll have a dedicated ECS integration for Elastic Agent at some point. I've prototyped one (elastic/integrations#2972) but it has some blockers. There seems to be movement on the blockers, though. Still, in the meantime it probably makes sense to document how to use the custom logs integration. The config would look very similar to the Filebeat config, just copy/pasted to the advanced section in the Elastic Agent integration.

[SylvainJuge]

@felixbarny do you know if there was any progress on the blockers you mentioned for a proper integration ?

Until we have a ready to use integration, I think it would still make sense to document using Elastic Agent based log ingestion, even if that requires a bit more effort than it would through a dedicated integration. The main issue I see for now is that people using Elastic Agent might think it's not supported, whereas it's just an alternate ingestion tool.

[felixbarny]

Yes, the blockers have been removed by we still don't have a dedicated ECS logging integration. +1 to adding docs for how to use Elastic Agent as a shipper in the meantime.

[bmorelli25]

I've added EA application logging to our Logs+ planning board in #2954.

[bmorelli25]

Moving to high priority. We'll get working on this after Oct. 31st.

[SylvainJuge]

Moving to high priority. We'll get working on this after Oct. 31st.

Good to know this is moving forward, if needed we have a few configuration examples that are available in the apm contrib repository: https://github.com/elastic/apm-contrib/tree/main/apm-agent-java/log-ingest, those are not "documentation level quality" but definitely relevant if needed to understand and experiment. Also we might add a new variant with the agent, or at least mention it once it's officially added to the documentation.

[bmorelli25]

The plan right now is to doc both Filebeat and Elastic Agent for self-managed/ESS users. We can't get Filebeat to work with Serverless (yet) so we'll only doc Elastic Agent with Serverless.

[bmorelli25]

Update: Filebeat works but will be much easier to use with Serverless in 8.11 because of this PR: elastic/beats#36649. We plan on doc'ing EA & Filebeat in both our classic docs and our Serverless docs.