Stabilizing trace injection into logs

[anuraaga]

I think we are in the midst of some large designs for logging support for OpenTelemetry, and it will take some time to work through designing it. I was wondering, would it be possible to stabilize the spec for injection of IDs, specifically this portion sooner?

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/overview.md#trace-context-in-legacy-formats

Injecting IDs into logs is very tried-and-trued, and in a sense seems to be relatively simple compared to many of the aspects of our spec. Stabilizing log injection of IDs would accelerate providing some correlation features for OpenTelemetry users with seemingly good bang for the buck.

[anuraaga]

@zenmoto @tigrannajaryan Wondering if you have any thoughts :)

[tigrannajaryan]

I would not want to rush one particular small part of the log spec, declare it stable and then tie our hands. Given that nothing is yet stable about logs I think it is better to wait on having a wider consensus on how logs should work before starting declaring individual bits of the spec stable.

We began declaring parts of the trace spec much later in the process, after tracing as a whole was scrutinized much more. I think that's the right approach for log spec too otherwise we may make premature decisions.

[anuraaga]

@tigrannajaryan I was wondering whether trace injection into legacy logs is actually an issue of the log spec though, and not actually tracing? Legacy logs and log context isn't a new thing, and injection is small scoped and something that's been used in tracing for a long time so it seems like just stabilizing a few key names could be quite helpful to tracing users.

[anuraaga]

For a bit more context, we're wondering whether we should even support injection of IDs into logs for OpenTelemetry tracing users. I feel we should, injecting tracing information into logs is tried-and-trued, improves observability a lot, and will be a blocker for tracing users migrating to OpenTelemetry if it's not well supported. Since tracing's 1.0, I suspect the intent is for tracing users to be able to start migrating - so it seems like this part of the spec could be considered for stabilization as part of the scope of tracing, not logs.

I've found that both Java, Python support it

https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/log4j/log4j-2.13.2/library
https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-logging

I think dotnet supports it via the framework itself

dotnet/aspnetcore#11211

These don't use our key names trace_id etc though, instead being TraceId.

I'm wondering what the plans are in other languages in terms of supporting injection of IDs into logs - any thoughts @MrAlias @dyladan or anyone else? I suspect there would be enough interest that this topic could be considered separately from the overall logs work.

[tigrannajaryan]

@anuraaga have you seen this OTEP? open-telemetry/oteps#150
It suggests a uniform approach for all languages, including context injection into logs.

[tigrannajaryan]

Before declaring the spec for trace_id stable I think it would be useful to:

• Align all implementations with the current spec (trace_id vs TraceId for .net).
• Wait for prototyping effort according to OTEP 0150 to complete and provide feedback on feasibility of trace context injection for variety of libraries and cases.

I do not feel comfortable declaring any parts of the log spec stable until we get more substantial feedback on OTEP 0150 and on implementing logs in general.

[alolita]

@jpkrohling @rakyll - thoughts on this proposal? I am hesitant to fully support trace ID injection into logs for OTel tracing users since the log spec is work in progress. This practice is used by some frameworks. Java, Python and .Net support this implementation currently as @anuraaga has verified above.

[jpkrohling]

This particular request seems simple enough, with only a couple of implementations that would need to be changed. As such, I think they don't have to wait for the spec to be ready to start doing what is written in the linked proposal. It just needs to be clearly labeled as experimental, as the field name might change by the time the spec is released, affecting current users of the libraries.

[tigrannajaryan]

Correct, I do not see the need to wait with implementing. I only see the need to wait with declaring the spec stable.

I would say that it is highly likely that what the spec says today with regards to trace context injection in logs will be eventually declared stable as-is (but we can't be sure so let's keep the opportunity to change open).