feat: Next.js server instrumentation

[trentm]

Closes: #1611

Planned commit message:

feat: Next.js server instrumentation

This adds instrumentation of the Next.js dev server (`next dev`) and prod
server (`next start`). The APM transactions for incoming HTTP requests to the
server will be named appropriately based on Next.js's routing -- both for
user page routes (e.g. `GET /a-dynamic-page/[id]`) and for internal Next.js
routes (e.g. `Next.js _next/data route my-page`,
`Next.js Rewrite route /foo -> /bar`). As well, exceptions in server-side code
(e.g. `getServerSideProps`, server-side run page handlers, API handlers) will
be reported.

This is a "technical preview", meaning we might change how this works in
future versions -- including dropping the separate "start-next.js" start
module.

This also makes a change to stacktrace reporting. If a stack frame refers to
a "*.min.js" file path and does not have source-map information, it is assumed
to be a minimized file and source file context is *not* reported. This is to
avoid excessively large stacktrace data. Reporting source file lines for
name-minimized and 500+ character lines is not useful.

Checklist

• Implement code
• Add tests
• Update TypeScript typings
• Update documentation
• Add CHANGELOG.asciidoc entry
• Commit message follows commit guidelines
• Remove the following "XXX" notes before merging. They are about pointing docs/examples to the feature branch until merged to main:

examples/nextjs/package.json
11:  "XXX": "change to 'main' branch for merge, and issue to update this on release",

docs/nextjs.asciidoc
50:NOTE: XXX Until this is merged, the example is at: <https://github.com/elastic/apm-agent-nodejs/tree/trentm/feat-nextjs/examples/nextjs/>
63:NOTE: XXX Until this is in a release use `npm install "elastic-apm-node@elastic/apm-agent-nodejs#trentm/feat-nextjs"`.

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-10-27T18:47:41.575+0000

• Duration: 23 min 11 sec

Test stats 🧪

Test	Results
Failed	0
Passed	322664
Skipped	0
Total	322664

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• run module tests for <modules> : Run TAV tests for one or more modules, where <modules> can be either a comma separated list of modules (e.g. memcached,redis) or the string literal ALL to test all modules

• run benchmark tests : Run the benchmark test only.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[trentm]

@bmorelli25 I'd love your feedback, if you have a chance, on the added "Get started with Next.js" doc. A current build of that is here: https://apm-agent-nodejs_2959.docs-preview.app.elstc.co/guide/en/apm/agent/nodejs/master/nextjs.html

@astorm Here are some possibly helpful review notes. Please ask if you have Qs.

• To play around with the instrumentation there is the simple "examples/nextjs/" app. However, a better play app is the one used for testing:

cd test/instrumentation/modules/next/a-nextjs-app
npm ci

# start a mockapmserver

npm run dev   # to run the dev server

npm run build && npm start  # to run the prod server

# Open http://localhost:3000/ in your browser and click around, or try some of these:
curl -i localhost:3000/a-page/
curl -i localhost:3000/a-page-redirect

curl -i localhost:3000/rewrite-to-a-page
curl -i localhost:3000/rewrite-to-a-dynamic-page
curl -i localhost:3000/rewrite-to-a-public-file
curl -i localhost:3000/rewrite-to-a-404
curl -i localhost:3000/rewrite-external/foo

curl -i localhost:3000/
curl -i localhost:3000/a-page
curl -i localhost:3000/a-dynamic-page/42
curl -i localhost:3000/an-ssr-page
curl -i localhost:3000/an-ssr-error-page

curl -i localhost:3000/api/an-api-endpoint
curl -i localhost:3000/api/a-dynamic-api-endpoint/123
curl -i localhost:3000/api/an-api-endpoint-that-throws

• "lib/instrumentation/modules/next/README.md" attempts to give an overview of how the instrumentation works. Reading "lib/instrumentation/modules/next/dist/server/next-server.js" is the main meat.
• For testing, "test/instrumentation/modules/next/next.test.js" is the main thing to read.

[bmorelli25]

I really like how these docs turned out. I only have a few suggestions.

[astorm]

Seems ok/good? I had a look over everything and left some questions but nothing seems out of line for a tech preview. I also ran through the basic docs (yay docs) and was able to get an example app up, running, instrumented with the agent and it produced reasonable transaction names.

I didn't go too deep into the specifics of which Next.js modules/methods we were instrumenting, so if there's something specific in the instrumented packages you think needs more scrutiny let me know, otherwise approved.

[astorm]

no objections, (and maybe it gets clearer if I get deeper into the review), but why this new .tav.yml file instead of using the main one?

[trentm]

I don't have an entirely solid answer. Mainly because it allows for clean separation of the package.json for this Next.js app. If this tav config was in the top-level .tav.yml file, then the dependencies:

  "dependencies": {
    "next": "^13.0.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0"
  }

would have to be in the top-level "devDependencies". In general that muddies the waters for understanding just this Next.js test case. Currently as it is you can work just in this test dir:

cd test/instrumentation/modules/next/a-nextjs-app
npm install
node ../next.test.js
npx tav

While doing (the long) dev on this PR that was helpful. And especially while learning Next.js it was helpful to be clear about exactly what deps were involved for the test app... and not the ~80 devDeps that we currently have at the top-level.

In this case there aren't any module conflicts but, for example, there are with the graphql-related modules (hence the npm uninstall ... commands in .tav.yml. I'm not proposing doing it soon, but I think it would be interesting to eventually consider a layout where there are multiple separate .tav.yml files -- perhaps one per instrumentation module.

[astorm]

XXX cleanup?

[trentm]

Yes, I have a checklist bullet for this in the description above.

[astorm]

XXX cleanup?

[trentm]

Yes, I have a checklist bullet for this in the description above.

[astorm]

XXX cleanup?

[trentm]

Yes, I have a checklist bullet for this in the description above.

[apmmachine]

❕ Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-10-28T20:05:26.118+0000

• Duration: 24 min 40 sec

Test stats 🧪

Test	Results
Failed	0
Passed	311021
Skipped	0
Total	311021

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• run module tests for <modules> : Run TAV tests for one or more modules, where <modules> can be either a comma separated list of modules (e.g. memcached,redis) or the string literal ALL to test all modules

• run benchmark tests : Run the benchmark test only.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)