Should response header fields be optional?

[dyladan]

#365 (comment)

[yurishkuro]

I think they certainly should be optional. We have two use cases involving 3 IDs: self trace ID, self span ID, and proposed parent ID. Both use cases themselves are optional and don't make sense for every service (e.g. for a service in the middle of your own architecture returning a response header is almost always a waste). I expect the two use cases to be at minimum configurable in the SDK, and perhaps even triggered by something in the request.

[dyladan]

I also believe they should be optional. Some have mentioned that making them optional adds complexity, but I think it is worthwhile.

[dyladan]

AFAIK @SergeyKanzhelev had some concern here?

[dyladan]

Discussed at the bi-weekly meeting. The discussion turned from "should they be optional" to "if they're optional, how do you decide which fields to include?"

I will attempt to make a set of rules for how to decide which fields to include. If a set of rules cannot be made reasonably and reliably, the fields cannot be optional.

[dyladan]

At last week's meeting, I volunteered to think about a set of rules that could be applied to determine which fields should be returned in the response header.

Uses of fields

Here is a quick summary of the fields available and their suggested uses.

Version

Ensure header is parsed correctly. Always required if the header is included in the response.

Trace ID

When called by a third party, this may be used to communicate the recorded trace ID to the caller for support purposes.

When called by a load balancer or web browser, this maybe be used to provide a trace ID to the caller so they may participate in the trace.

Proposed Parent Id

When called by a load balancer or web browser, this maybe be used to provide a span ID to the caller so they may participate in the trace.

Trace Flags

When called by a third party, this may be used to communicate information about the trace such as the sampled status for support purposes.

When called by a load balancer, this may be used to communicated information such as the sampled status so that the caller may intelligently decide to record its part of the trace or not.

Suggested decision model

Of course, the header itself and all fields (except version) are always optional. If a service provider wishes not to return the response header for any reason, they are not obligated to do so. The following rules are only a recommended set of rules to encourage interoperation between tracing providers.

Default behavior

Incoming call with traceparent

If the trace ID of the traceparent is used, do not include the header in the response.

If a new trace ID is used (trace restarted), include the version and the trace ID in the response.

Incoming call without traceparent

Return version, trace ID, and trace flags.

When configured as an internal server-to-server service with no load balancer (or a load balancer which does not participate in the trace).

Do not include response headers. The trace can be successful with use of only the request headers.

When configured as an internal server-to-server service behind a tracing load balancer

Include version and trace flags in the response so the caller may respect the called service's sampling decision.

When configured as a web server

On initial document load only (index.html), include version, trace ID, proposed parent ID, and trace flags. If a web browser participates in the trace, it may use the trace ID as its own trace ID, and the proposed parent id as its own span ID.

Known Problems

There is currently no API available to JavaScript in the browser which would allow a tracing agent to access the response header. An API would need to be implemented which allows access to the full header, or its parsed values. Alternatively, the web browser could include traceparent request headers on requests (possibly configurable by some HTML meta tag or other mechanism). This would remove the need for the response header in the browser use-case.

Operating "modes" will need to be configured by the application owner or operator.

The web browser proposed parent use case would require tracing providers to be resilient to missing root spans in the case that the first 'hop' generates a proposed span id, but the browser for some reason does not report a span with that id.

[yurishkuro]

@dyladan great summary. I assume this still leaves an open question of the header format (my preference is a KV list given the fields optionality: v=1;tid=***;ppid=***;flags=**).

[dyladan]

@yurishkuro at this point I think everything is an open question. @SergeyKanzhelev has raised concerns that optional fields will cause confusion with the traceparent header, which has no optional fields. Having a completely different format like KV could alleviate this problem.

[dyladan]

At the workshop we discussed making all fields required. Given the "known problems" linked above, it was decided to remove the proposed parent ID from the response header. In its place, the span id of the child will be included.

All fields were deemed useful for the support use-case and fixed length simplifies generating and parsing the header. @yurishkuro you were the only person who interacted with this issue, so do you have an objection to making fields all required given the changed fields?

[yurishkuro]

sgtm

[dyladan]

Changes discussed (all fields required) were made in #443

[kalyanaj]

The group already decided and resolved that the entire response header is optional, but all fields in it were mandatory.