[feature request] support extensible tag serialization

[djluck]

Package

OpenTelemetry.Exporter.OpenTelemetryProtocol

Is your feature request related to a problem?

It would be useful to better support the serialization of types not directly supported by the OTel protocol to tags.

Currently types not supported in TagWriter.TryWriteTag are converted to strings. While this is better than nothing, we loose opportunities to convert objects into rich key/ value pairs. Such an approach would lend itself well to supporting wide events as users would no-longer have to serialize objects to tags by hand and could instead rely upon library code to do this.

The motivation for such an enhancement is discussed here: #5718.
The difference between this issue and the prior linked issue is that instead of asking this project to build + support such a serializer, this issue asks that TagWriter is simply made to be extensible, in order to support any custom serialization of types not supported natively by Otel such as:

• MyObject
• List<MyObject>
• DateTime
• etc.

What is the expected behavior?

I would expect the changes might be of this form:

• A public interface is extracted from TagWriter
• A convenient way is provided to override the default TagWriter instance

Which alternative solutions or features have you considered?

The alternative is that you can provide an extension method to support serialization of unsupported objects. But this has several drawbacks around performance:

• Adding tags to Activity is relatively slow, due to casting/ boxing of structs and acquiring/ releasing the mutex that synchronizes adding tags
• Boxing of struct types to object incurs additional allocations
• Converting objects to strings ahead of time doesn't allow for the pooling of buffers, further increasing memory allocations

Additional context

No response

[CodeBlanch]

When it comes to tracing, this needs to go through the spec.

Because the attributes are strictly defined to a very limited set of types:

An Attribute is a key-value pair, which MUST have the following properties:

• The attribute key MUST be a non-null and non-empty string.
  • Case sensitivity of keys is preserved. Keys that differ in casing are treated as distinct keys.
• The attribute value is either:
  • A primitive type: string, boolean, double precision floating point (IEEE 754-1985) or signed 64 bit integer.
  • An array of primitive type values. The array MUST be homogeneous,
    i.e., it MUST NOT contain values of different types.

What OTLP is doing is conforming to the spec and normalizing the objects it is fed.

I believe this also applies to metrics as well. I couldn't find something explicit in the spec. @alanwest?

When it comes to logs, things are a bit more interesting. The spec allows for attributes on logs to be AnyValue so deep graphs should be possible. But there are other issues there. On LogRecord we have Attributes property which is a list of KeyValuePair<string, object?>s. If a LogRecord is added to a batch, we copy the top-level key value pairs to pool storage. But we don't do a deep copy. In that regard, deep object graphs are NOT supported currently by OTel.NET. Because they may have mutated or have been disposed by the time the batch is exported. A solution in OTLP Exporter wouldn't alleviate this. Something would have to be done in the SDK to serialize these deep graphs onto the pool first so they are safe to later export.

[djluck]

Hey @CodeBlanch, thanks for taking the time to respond.

So you're absolutely correct about the spec- there are strict limits on what value types are supported. But we could still encode tree-like object structures with these limited sets of attribute value types, e.g.:

• For each property on an object, extract this as a new attribute key
• If the property type matches a supported attribute value types, map it to that types. If not, convert it to a string

I understand that this additional mapping is not outlined within the spec. So the ask is to allow users to be able to hook into the tag attribute serialization code, to handle the object case as they see fit.