Standalone ASM configuration and span tags #4291

iunanua · 2024-05-09T10:00:41Z

What does this PR do?

Add DD_EXPERIMENTAL_APPSEC_STANDALONE_ENABLED env var and experimental.appsec.standalone.enabled config option
If standalone is enabled, subscribe to dd-trace:span:start channel and add the _dd.apm.enabled:0 tag in root spans and child spans with remote parent.
include _dd.p.appsec trace tag whether an appsec or iast event occurs.

Motivation

This is the first part to support Standalone ASM billing. We have divided the feature into two PRs in order to facilitate the review.

Note for APM reviewers:

opentracing/span.js modified to include the fields param in dd-trace:span:start events
proxy.js modified to configure standalone module when initializing the tracer.

Plugin Checklist

Additional Notes

github-actions · 2024-05-09T10:01:23Z

Overall package size

Self size: 6.72 MB
Deduped: 61.97 MB
No deduping: 62.25 MB

Dependency sizes

name	version	self size	total size
@datadog/native-appsec	8.0.1	15.59 MB	15.6 MB
@datadog/native-iast-taint-tracking	2.1.0	14.91 MB	14.92 MB
@datadog/pprof	5.3.0	9.85 MB	10.22 MB
protobufjs	7.2.5	2.77 MB	6.56 MB
@datadog/native-iast-rewriter	2.3.1	2.15 MB	2.24 MB
@opentelemetry/core	1.14.0	872.87 kB	1.47 MB
@datadog/native-metrics	2.0.0	898.77 kB	1.3 MB
@opentelemetry/api	1.8.0	1.21 MB	1.21 MB
import-in-the-middle	1.7.4	70.19 kB	739.86 kB
msgpack-lite	0.1.26	201.16 kB	281.59 kB
opentracing	0.14.7	194.81 kB	194.81 kB
semver	7.5.4	93.4 kB	123.8 kB
pprof-format	2.1.0	111.69 kB	111.69 kB
@datadog/sketches-js	2.1.0	109.9 kB	109.9 kB
lodash.sortby	4.7.0	75.76 kB	75.76 kB
lru-cache	7.14.0	74.95 kB	74.95 kB
ignore	5.2.4	51.22 kB	51.22 kB
int64-buffer	0.1.10	49.18 kB	49.18 kB
shell-quote	1.8.1	44.96 kB	44.96 kB
istanbul-lib-coverage	3.2.0	29.34 kB	29.34 kB
tlhunter-sorted-set	0.1.0	24.94 kB	24.94 kB
limiter	1.1.5	23.17 kB	23.17 kB
dc-polyfill	0.1.4	23.1 kB	23.1 kB
retry	0.13.1	18.85 kB	18.85 kB
jest-docblock	29.7.0	8.99 kB	12.76 kB
crypto-randomuuid	1.0.0	11.18 kB	11.18 kB
path-to-regexp	0.1.7	6.78 kB	6.78 kB
koalas	1.0.2	6.47 kB	6.47 kB
module-details-from-path	1.0.3	4.47 kB	4.47 kB

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

packages/dd-trace/src/config.js

packages/dd-trace/test/opentracing/span.spec.js

packages/dd-trace/src/config.js

pr-commenter · 2024-05-13T12:09:28Z

Benchmarks

Benchmark execution time: 2024-06-18 08:20:11

Comparing candidate commit 7dc9981 in PR branch igor/standalone-asm-config-and-tags with baseline commit 3b479d0 in branch master.

Found 2 performance improvements and 0 performance regressions! Performance is the same for 258 metrics, 6 unstable metrics.

scenario:plugin-graphql-with-depth-and-collapse-on-18

🟩 max_rss_usage [-137.257MB; -134.923MB] or [-14.142%; -13.902%]

scenario:plugin-graphql-with-depth-on-max-18

🟩 max_rss_usage [-160.552MB; -98.832MB] or [-16.711%; -10.287%]

codecov · 2024-05-20T12:46:56Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.19%. Comparing base (5278b1c) to head (c37423a).
Report is 4 commits behind head on master.

❗ Current head c37423a differs from pull request most recent head 7dc9981

Please upload reports for the commit 7dc9981 to get more accurate results.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4291       +/-   ##
===========================================
- Coverage   80.42%   69.19%   -11.24%     
===========================================
  Files           3        1        -2     
  Lines         373      198      -175     
  Branches       33       33               
===========================================
- Hits          300      137      -163     
+ Misses         73       61       -12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

packages/dd-trace/src/opentracing/span.js

packages/dd-trace/src/config.js

CarlesDD

Missing type for experimental.appsec.standalone.enabled in index.d.ts and test.ts

packages/dd-trace/src/config.js

packages/dd-trace/src/opentracing/span.js

packages/dd-trace/test/appsec/iast/vulnerability-reporter.spec.js

packages/dd-trace/test/opentracing/tracer.spec.js

Co-authored-by: Ugaitz Urien <[email protected]>

packages/dd-trace/test/opentracing/span.spec.js

packages/dd-trace/test/config.spec.js

simon-id · 2024-06-18T16:28:38Z

packages/dd-trace/src/proxy.js

@@ -180,6 +181,7 @@ class Tracer extends NoopProxy {
      if (!this._tracingInitialized) {
        this._tracer = new DatadogTracer(config)
        this.appsec = new AppsecSdk(this._tracer, config)
+        appsecStandalone.configure(config)


maybe we could move this configure() somewhere in appsec territory ? it's not super important but if we can do it in a clean way it could be nice

I need it in proxy.js to replace the default PrioritySampler via DatadogTracer constructor, something you could see in the next standalone PR.
Also standalone is not related with AppsecSdk so I'd prefer not to mix the two features.

* DD_APM_TRACING_ENABLED and span _dd.apm.enabled tag * clean up * Use MANUAL_KEEP const * Add _dd.p.appsec tag on standalone ASM events * Include apmTracingEnabled checks * Appsec Reporter tests * Appsec sdk track_event test * Use numeric value for _dd.p.appsec * Include appsec standalone config in .ts files * Clean up null and undefined values * Remove not needed config properties * standalone module * Clean up * standalone proxy test * Update packages/dd-trace/test/appsec/iast/vulnerability-reporter.spec.js Co-authored-by: Ugaitz Urien <[email protected]> * appsec reporter test * Use standalone singletone in vulnerability-reporter * continue applying ratelimiter on appsec standalone events * Update packages/dd-trace/src/appsec/reporter.js Co-authored-by: simon-id <[email protected]> * Add _dd.apm.enabled:0 in root spans with remote parent * Use a method to add the tag * Remove apmTracingEnabled config property * Add _dd.p.appsec tag in trace tags * Some tests * Set _dd.apm.enabled in root span * configure standalone if _tracingInitialized * Use dd-trace:span:start channel * Clean up * use a meta tag * hasSubscribers check * test description * Check span context has tags before using them and check if config has changed * clean up * Clean up * Clean up * clean up * Update packages/dd-trace/test/appsec/sdk/track_event.spec.js Co-authored-by: Ugaitz Urien <[email protected]> * Update packages/dd-trace/test/appsec/standalone.spec.js Co-authored-by: Ugaitz Urien <[email protected]> * protect sample method * Use assert instead expect * unsubscribe after test * suggestions --------- Co-authored-by: Ugaitz Urien <[email protected]> Co-authored-by: simon-id <[email protected]>

DD_APM_TRACING_ENABLED and span _dd.apm.enabled tag

f14601e

iunanua added the semver-minor label May 9, 2024

iunanua commented May 10, 2024

View reviewed changes

packages/dd-trace/src/config.js Outdated Show resolved Hide resolved

iunanua commented May 10, 2024

View reviewed changes

packages/dd-trace/src/config.js Outdated Show resolved Hide resolved

CarlesDD reviewed May 13, 2024

View reviewed changes

packages/dd-trace/test/opentracing/span.spec.js Outdated Show resolved Hide resolved

simon-id reviewed May 13, 2024

View reviewed changes

packages/dd-trace/src/config.js Outdated Show resolved Hide resolved

clean up

2817602

Use MANUAL_KEEP const

957760a

iunanua force-pushed the igor/standalone-asm-config-and-tags branch 2 times, most recently from ff3519d to 2bb1295 Compare May 17, 2024 13:45

Add _dd.p.appsec tag on standalone ASM events

5d6210d

iunanua force-pushed the igor/standalone-asm-config-and-tags branch from 2bb1295 to 5d6210d Compare May 20, 2024 09:30

Include apmTracingEnabled checks

089c26d

iunanua added 3 commits May 20, 2024 15:26

Appsec Reporter tests

18d9fef

Appsec sdk track_event test

d903394

Use numeric value for _dd.p.appsec

736014a

iunanua commented May 20, 2024

View reviewed changes

packages/dd-trace/src/opentracing/span.js Outdated Show resolved Hide resolved

iunanua commented May 20, 2024

View reviewed changes

packages/dd-trace/src/config.js Outdated Show resolved Hide resolved

iunanua changed the title ~~Standalone ASM part I~~ Standalone ASM configuration and span tags May 21, 2024

CarlesDD reviewed May 21, 2024

View reviewed changes

uurien reviewed May 21, 2024

View reviewed changes

packages/dd-trace/src/config.js Outdated Show resolved Hide resolved

packages/dd-trace/src/opentracing/span.js Outdated Show resolved Hide resolved

uurien reviewed May 21, 2024

View reviewed changes

packages/dd-trace/test/appsec/iast/vulnerability-reporter.spec.js Outdated Show resolved Hide resolved

uurien reviewed May 21, 2024

View reviewed changes

packages/dd-trace/test/opentracing/tracer.spec.js Outdated Show resolved Hide resolved

iunanua added 2 commits May 21, 2024 15:56

Include appsec standalone config in .ts files

396a8fc

Clean up null and undefined values

cb73c58

iunanua marked this pull request as ready for review May 22, 2024 13:48

iunanua requested a review from a team as a code owner May 22, 2024 13:48

iunanua and others added 3 commits June 12, 2024 11:52

Update packages/dd-trace/test/appsec/standalone.spec.js

888c81f

Co-authored-by: Ugaitz Urien <[email protected]>

protect sample method

58a9c6a

Use assert instead expect

99675fb

uurien reviewed Jun 12, 2024

View reviewed changes

packages/dd-trace/test/opentracing/span.spec.js Show resolved Hide resolved

uurien previously approved these changes Jun 12, 2024

View reviewed changes

uurien self-requested a review June 12, 2024 16:01

unsubscribe after test

2a03d12

iunanua dismissed uurien’s stale review via 2a03d12 June 13, 2024 07:12

uurien previously approved these changes Jun 13, 2024

View reviewed changes

iunanua added 2 commits June 13, 2024 10:13

Merge branch 'master' into igor/standalone-asm-config-and-tags

0547b74

Merge branch 'master' into igor/standalone-asm-config-and-tags

0b82fe8

simon-id reviewed Jun 17, 2024

View reviewed changes

packages/dd-trace/test/opentracing/span.spec.js Outdated Show resolved Hide resolved

simon-id reviewed Jun 17, 2024

View reviewed changes

packages/dd-trace/test/config.spec.js Show resolved Hide resolved

iunanua added 2 commits June 18, 2024 10:08

Merge branch 'master' into igor/standalone-asm-config-and-tags

999f136

suggestions

7dc9981

iunanua dismissed uurien’s stale review via 7dc9981 June 18, 2024 08:10

simon-id approved these changes Jun 18, 2024

View reviewed changes

uurien approved these changes Jun 18, 2024

View reviewed changes

simon-id reviewed Jun 18, 2024

View reviewed changes

iunanua merged commit b1f1f85 into master Jun 19, 2024
134 checks passed

iunanua deleted the igor/standalone-asm-config-and-tags branch June 19, 2024 13:36

iunanua mentioned this pull request Jun 19, 2024

Standalone ASM priority sampler and tag propagation #4416

Merged

6 tasks

This was referenced Jul 10, 2024

v5.18.0 #4496

Merged

v4.42.0 #4497

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Standalone ASM configuration and span tags #4291

Standalone ASM configuration and span tags #4291

iunanua commented May 9, 2024 •

edited

Loading

github-actions bot commented May 9, 2024 •

edited

Loading

pr-commenter bot commented May 13, 2024 •

edited

Loading

codecov bot commented May 20, 2024 •

edited

Loading

CarlesDD left a comment

simon-id Jun 18, 2024

iunanua Jun 19, 2024 •

edited

Loading

Standalone ASM configuration and span tags #4291

Standalone ASM configuration and span tags #4291

Conversation

iunanua commented May 9, 2024 • edited Loading

What does this PR do?

Motivation

Note for APM reviewers:

Plugin Checklist

Additional Notes

github-actions bot commented May 9, 2024 • edited Loading

Overall package size

Dependency sizes

pr-commenter bot commented May 13, 2024 • edited Loading

Benchmarks

scenario:plugin-graphql-with-depth-and-collapse-on-18

scenario:plugin-graphql-with-depth-on-max-18

codecov bot commented May 20, 2024 • edited Loading

Codecov Report

CarlesDD left a comment

Choose a reason for hiding this comment

simon-id Jun 18, 2024

Choose a reason for hiding this comment

iunanua Jun 19, 2024 • edited Loading

Choose a reason for hiding this comment

iunanua commented May 9, 2024 •

edited

Loading

github-actions bot commented May 9, 2024 •

edited

Loading

pr-commenter bot commented May 13, 2024 •

edited

Loading

codecov bot commented May 20, 2024 •

edited

Loading

iunanua Jun 19, 2024 •

edited

Loading