Update to newer version of .NET SDK #6585

andrewlock · 2025-01-22T16:35:39Z

Summary of changes

Bump the version of the SDK we use to build

Reason for change

The 9.0.0 includes possible seg faults, we're hoping this will reduce random flake

Implementation details

Bump from 9.0.100 -> 9.0.102

Test coverage

Did a test with the updated gitlab build image here and looks OK, will revert the temporary change.

Other details

The VMs will need updating after this + we'll need to update the gitlab build image

datadog-ddstaging · 2025-01-22T17:01:59Z

Datadog Report

Branch report: andrew/ci/bump-dotnet-sdk
Commit report: e1cb41d
Test service: dd-trace-dotnet

✅ 0 Failed, 245528 Passed, 2004 Skipped, 18h 39m 58.61s Total Time

andrewlock · 2025-01-22T17:25:34Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (69ms)  : 67, 71
     .   : milestone, 69,
    master - mean (69ms)  : 66, 72
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (981ms)  : 962, 1000
     .   : milestone, 981,
    master - mean (985ms)  : 961, 1010
     .   : milestone, 985,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (108ms)  : 106, 110
     .   : milestone, 108,
    master - mean (109ms)  : 106, 111
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (679ms)  : 662, 695
     .   : milestone, 679,
    master - mean (684ms)  : 667, 701
     .   : milestone, 684,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (92ms)  : 90, 93
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (634ms)  : 621, 648
     .   : milestone, 634,
    master - mean (635ms)  : 619, 650
     .   : milestone, 635,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (189ms)  : 186, 193
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (1,086ms)  : 1046, 1125
     .   : milestone, 1086,
    master - mean (1,087ms)  : 1062, 1113
     .   : milestone, 1087,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (276ms)  : 272, 281
     .   : milestone, 276,
    master - mean (275ms)  : 271, 280
     .   : milestone, 275,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (870ms)  : 837, 904
     .   : milestone, 870,
    master - mean (874ms)  : 845, 903
     .   : milestone, 874,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6585) - mean (264ms)  : 261, 267
     .   : milestone, 264,
    master - mean (265ms)  : 262, 268
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6585) - mean (842ms)  : 809, 876
     .   : milestone, 842,
    master - mean (854ms)  : 823, 884
     .   : milestone, 854,

This reverts commit 53d6e2a.

andrewlock · 2025-01-22T19:46:27Z

Benchmarks Report for tracer 🐌

Benchmarks for #6585 compared to master:

1 benchmarks are faster, with geometric mean 1.123
3 benchmarks are slower, with geometric mean 1.159
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.93μs	34.4ns	150ns	0.0159	0.00396	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.1μs	46.5ns	180ns	0.0147	0.00491	0	5.81 KB
master	`StartStopWithChild`	net472	16.4μs	47.6ns	184ns	1.04	0.296	0.0986	6.21 KB
#6585	`StartStopWithChild`	net6.0	8.03μs	43ns	243ns	0.0117	0.00778	0	5.61 KB
#6585	`StartStopWithChild`	netcoreapp3.1	9.87μs	54ns	333ns	0.0152	0.00506	0	5.8 KB
#6585	`StartStopWithChild`	net472	16.2μs	49.2ns	191ns	1.05	0.309	0.0975	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	465μs	317ns	1.23μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	293ns	1.13μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	861μs	342ns	1.28μs	0.428	3.3 KB
#6585	`WriteAndFlushEnrichedTraces`	net6.0	504μs	352ns	1.27μs	0	2.7 KB
#6585	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	653μs	342ns	1.33μs	0	2.7 KB
#6585	`WriteAndFlushEnrichedTraces`	net472	844μs	599ns	2.32μs	0.419	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	130μs	446ns	1.73μs	0.132	14.47 KB
master	`SendRequest`	netcoreapp3.1	146μs	453ns	1.75μs	0.219	17.27 KB
master	`SendRequest`	net472	0.0127ns	0.00368ns	0.0143ns	0	0 b
#6585	`SendRequest`	net6.0	128μs	622ns	2.64μs	0.188	14.47 KB
#6585	`SendRequest`	netcoreapp3.1	141μs	461ns	1.79μs	0.21	17.27 KB
#6585	`SendRequest`	net472	0.0124ns	0.00372ns	0.0144ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6585

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	41.49 KB	41.86 KB	377 B	0.91%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	586μs	3.04μs	14.6μs	0.573	0	0	41.69 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	711μs	3.94μs	24μs	0.351	0	0	41.49 KB
master	`WriteAndFlushEnrichedTraces`	net472	869μs	4.42μs	19.3μs	8.19	2.59	0.431	53.31 KB
#6585	`WriteAndFlushEnrichedTraces`	net6.0	555μs	2.69μs	10.8μs	0.543	0	0	41.57 KB
#6585	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	648μs	3.24μs	14.9μs	0.321	0	0	41.86 KB
#6585	`WriteAndFlushEnrichedTraces`	net472	834μs	3.81μs	14.8μs	8.39	2.52	0.419	53.24 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.26μs	1.17ns	4.51ns	0.0145	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.79μs	2.25ns	8.73ns	0.0134	0	1.02 KB
master	`ExecuteNonQuery`	net472	1.95μs	2.3ns	8.9ns	0.156	0.000976	987 B
#6585	`ExecuteNonQuery`	net6.0	1.25μs	0.928ns	3.6ns	0.0144	0	1.02 KB
#6585	`ExecuteNonQuery`	netcoreapp3.1	1.77μs	0.792ns	2.74ns	0.0133	0	1.02 KB
#6585	`ExecuteNonQuery`	net472	2.1μs	1.89ns	7.33ns	0.156	0.00105	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.3μs	0.645ns	2.41ns	0.0138	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.55μs	1.2ns	4.63ns	0.0132	976 B
master	`CallElasticsearch`	net472	2.53μs	1.93ns	7.46ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.33μs	0.594ns	2.22ns	0.0134	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.6μs	0.998ns	3.73ns	0.0135	1.02 KB
master	`CallElasticsearchAsync`	net472	2.69μs	1.18ns	4.27ns	0.166	1.05 KB
#6585	`CallElasticsearch`	net6.0	1.22μs	0.588ns	2.2ns	0.0135	976 B
#6585	`CallElasticsearch`	netcoreapp3.1	1.5μs	0.748ns	2.59ns	0.0129	976 B
#6585	`CallElasticsearch`	net472	2.59μs	1.63ns	6.11ns	0.157	995 B
#6585	`CallElasticsearchAsync`	net6.0	1.41μs	0.612ns	2.29ns	0.0134	952 B
#6585	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	1.81ns	7ns	0.0139	1.02 KB
#6585	`CallElasticsearchAsync`	net472	2.68μs	1.28ns	4.96ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.34μs	0.474ns	1.77ns	0.0135	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.58μs	1.62ns	6.07ns	0.0126	952 B
master	`ExecuteAsync`	net472	1.86μs	0.532ns	2.06ns	0.145	915 B
#6585	`ExecuteAsync`	net6.0	1.25μs	1.03ns	3.84ns	0.0133	952 B
#6585	`ExecuteAsync`	netcoreapp3.1	1.58μs	2.3ns	8.59ns	0.0127	952 B
#6585	`ExecuteAsync`	net472	1.9μs	0.583ns	2.18ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.35μs	2.24ns	8.39ns	0.0329	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.33μs	2.03ns	7.85ns	0.0373	2.85 KB
master	`SendAsync`	net472	7.34μs	1.84ns	7.11ns	0.495	3.12 KB
#6585	`SendAsync`	net6.0	4.5μs	0.982ns	3.68ns	0.0315	2.31 KB
#6585	`SendAsync`	netcoreapp3.1	5.29μs	2.69ns	10.4ns	0.0368	2.85 KB
#6585	`SendAsync`	net472	7.44μs	1.11ns	4.14ns	0.494	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.43μs	1.14ns	4.25ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.23μs	0.685ns	2.47ns	0.0222	1.64 KB
master	`EnrichedLog`	net472	2.54μs	1.49ns	5.77ns	0.25	1.57 KB
#6585	`EnrichedLog`	net6.0	1.47μs	2.54ns	9.86ns	0.0229	1.64 KB
#6585	`EnrichedLog`	netcoreapp3.1	2.28μs	1.12ns	4.06ns	0.0216	1.64 KB
#6585	`EnrichedLog`	net472	2.59μs	0.752ns	2.81ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	116μs	112ns	433ns	0.0579	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	254ns	951ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	142ns	549ns	0.684	0.228	4.46 KB
#6585	`EnrichedLog`	net6.0	116μs	142ns	551ns	0.0578	0	4.28 KB
#6585	`EnrichedLog`	netcoreapp3.1	120μs	225ns	872ns	0	0	4.28 KB
#6585	`EnrichedLog`	net472	150μs	76.6ns	265ns	0.675	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.98μs	1.02ns	3.94ns	0.0309	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.13μs	2.63ns	10.2ns	0.0289	2.2 KB
master	`EnrichedLog`	net472	4.98μs	1.33ns	5.13ns	0.32	2.02 KB
#6585	`EnrichedLog`	net6.0	3.09μs	0.903ns	3.5ns	0.0313	2.2 KB
#6585	`EnrichedLog`	netcoreapp3.1	4.26μs	1.3ns	4.87ns	0.0297	2.2 KB
#6585	`EnrichedLog`	net472	4.84μs	1.64ns	6.34ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.32μs	0.755ns	2.92ns	0.0158	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.81μs	0.395ns	1.42ns	0.0153	1.14 KB
master	`SendReceive`	net472	2.22μs	1.12ns	4.33ns	0.183	1.16 KB
#6585	`SendReceive`	net6.0	1.36μs	0.851ns	3.3ns	0.0164	1.14 KB
#6585	`SendReceive`	netcoreapp3.1	1.72μs	0.619ns	2.32ns	0.0155	1.14 KB
#6585	`SendReceive`	net472	2.17μs	3.14ns	11.7ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.82μs	1.18ns	4.57ns	0.0226	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.81μs	1.45ns	5.63ns	0.021	1.65 KB
master	`EnrichedLog`	net472	4.31μs	6.23ns	24.1ns	0.322	2.04 KB
#6585	`EnrichedLog`	net6.0	2.83μs	0.872ns	3.38ns	0.0213	1.6 KB
#6585	`EnrichedLog`	netcoreapp3.1	4μs	1.28ns	4.78ns	0.022	1.65 KB
#6585	`EnrichedLog`	net472	4.54μs	2.67ns	10.3ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6585

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.194	688.85	822.63
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472	1.163	598.60	696.15

Faster 🎉 in #6585

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.123	452.77	403.02

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	452ns	0.614ns	2.38ns	0.00813	576 B
master	`StartFinishSpan`	netcoreapp3.1	564ns	0.835ns	3.01ns	0.00773	576 B
master	`StartFinishSpan`	net472	600ns	1.25ns	4.84ns	0.0918	578 B
master	`StartFinishScope`	net6.0	536ns	0.73ns	2.83ns	0.00984	696 B
master	`StartFinishScope`	netcoreapp3.1	687ns	1.67ns	6.48ns	0.00963	696 B
master	`StartFinishScope`	net472	868ns	1.77ns	6.84ns	0.104	658 B
#6585	`StartFinishSpan`	net6.0	403ns	0.471ns	1.82ns	0.00798	576 B
#6585	`StartFinishSpan`	netcoreapp3.1	593ns	0.973ns	3.64ns	0.00783	576 B
#6585	`StartFinishSpan`	net472	696ns	1.59ns	6.15ns	0.0918	578 B
#6585	`StartFinishScope`	net6.0	492ns	0.674ns	2.43ns	0.0099	696 B
#6585	`StartFinishScope`	netcoreapp3.1	823ns	1.6ns	6.18ns	0.00913	696 B
#6585	`StartFinishScope`	net472	835ns	1.44ns	5.59ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6585

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.121	851.90	955.16

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	626ns	0.918ns	3.56ns	0.00978	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	853ns	1.3ns	5.02ns	0.00942	696 B
master	`RunOnMethodBegin`	net472	1.02μs	2ns	7.73ns	0.104	658 B
#6585	`RunOnMethodBegin`	net6.0	619ns	1.46ns	5.66ns	0.0099	696 B
#6585	`RunOnMethodBegin`	netcoreapp3.1	954ns	1.62ns	6.28ns	0.00953	696 B
#6585	`RunOnMethodBegin`	net472	1.08μs	1.45ns	5.62ns	0.104	658 B

Update to newer version of .NET SDK

339a6fd

andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 22, 2025

andrewlock requested review from a team as code owners January 22, 2025 16:35

andrewlock added 3 commits January 22, 2025 17:52

Update gitlab docker image

53d6e2a

TEMP use test docker image from docker hub

5799048

Revert "Update gitlab docker image"

e1cb41d

This reverts commit 53d6e2a.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to newer version of .NET SDK #6585

Update to newer version of .NET SDK #6585

andrewlock commented Jan 22, 2025 •

edited

Loading

datadog-ddstaging bot commented Jan 22, 2025 •

edited

Loading

andrewlock commented Jan 22, 2025 •

edited

Loading

andrewlock commented Jan 22, 2025

Raw results

Raw results

Raw results

More allocations ⚠️ in #6585

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6585

Faster 🎉 in #6585

Raw results

Slower ⚠️ in #6585

Raw results

Update to newer version of .NET SDK #6585

Are you sure you want to change the base?

Update to newer version of .NET SDK #6585

Conversation

andrewlock commented Jan 22, 2025 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Jan 22, 2025 • edited Loading

Datadog Report

andrewlock commented Jan 22, 2025 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jan 22, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #6585

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6585

Faster 🎉 in #6585

Raw results

Slower ⚠️ in #6585

Raw results

andrewlock commented Jan 22, 2025 •

edited

Loading

datadog-ddstaging bot commented Jan 22, 2025 •

edited

Loading

andrewlock commented Jan 22, 2025 •

edited

Loading