feat(instrumentation-express): propagate context and measure full handler spans

[raphael-theriault-swi]

Which problem is this PR solving?

Currently the Express instrumentation does not propagate context and does not properly instrument request handlers.

Short description of the changes

Context is propagated for middleware and request handler spans, while their next callback is reset to the parent context to avoid extreme nesting as suggested in #2022. Request handler spans are also no longer ended prematurely.

Router spans are kept as-is since they are broken and propagating context for them would make things even more confusing than they already are.

Example traces

Before

After

Related issues

• Fixes @opentelemetry/instrumentation-express handler does not run in correct context #2022
• Fixes @opentelemetry/instrumentation-express handler does not account for async work #2023
• Fixes Auto instrumentation of express app is not sending exception event in trace #2146
• Fixes express instrumentation - span end prematurely for non-middleware instrumentations #2442
• Fixes Can't update span name when using OpenTelemetry Express Instrumentation for Node.js #2469
• Fixes Can't get active span in Express instrumentation. #2617
• Supersedes fix(instrumentation-express): Finish spans after execution #2603

[raphael-theriault-swi]

cc @pichlermarc fixes discussed wednesday

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.97%. Comparing base (b641e1b) to head (5c016d9).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2638   +/-   ##
=======================================
  Coverage   90.96%   90.97%           
=======================================
  Files         172      172           
  Lines        8137     8140    +3     
  Branches     1649     1649           
=======================================
+ Hits         7402     7405    +3     
  Misses        735      735           

Files with missing lines	Coverage Δ
...try-instrumentation-express/src/instrumentation.ts	98.61% <100.00%> (+0.02%)	⬆️

[pichlermarc]

this looks good, thank you for working on this. 🙌
cc @JamieDanielson and @pkanal (component owners) for comments :)

[mikesir87]

Apologies for the outsider coming in on this PR, but reporting that I've manually applied this PR into a local working copy to validate it works with my sample apps and it worked! In fact, it already helped me catch a case or two where I was missing an await call on a Kafka publish or DB update that was running after the request handler had returned. Excited to see this contribution! 😄

[faranak-cs]

Waiting for the merge... 🤞

[dyladan]

Can you expand on what you mean by not working properly or create an issue so this doesn't get lost?

[raphael-theriault-swi]

You can see in the nested routers test that the created router spans never include the /posts section of the route. If I were to propagate context and nest them we'd end up with /api/user/:id -> /:postId -> /api/user/posts/:postId which is really confusing.