Add smoke tests for fleet installer #6645

andrewlock · 2025-02-10T11:46:30Z

Summary of changes

Adds a new smoke test stage, for running inside Windows using the fleet installer

Reason for change

Emulates an end-to-end smoke test for the fleet installer executable.

Implementation details

Build the fleet installer in the AzDo pipeline, as part of the package-windows stage
Adds a new fleet_installer_smoke_tests stage for running the smoke tests
Fix issues in ASP.NET Core test app, so that it works when hosted in IIS
Add a dockerfile for running the ASP.NET Core app with the fleet installer, hosted in IIS

That latter point proved to be somewhat of a nightmare. The resulting dockerfile feels kind of horrible, but I couldn't find another way to have:

Delay the start of the app pool until the container image is running (i.e. not during build time)
Start the worker process when the app pool starts, without an incoming request
Shut down the app pool when the worker process exits (which is the way the aspnetcore app works)
Exit the container

The workaround, using the entrypoint script, makes a 404 request to the app to spin up the worker process, manually shuts down the app pool (to make sure the app definitely shuts down and flushes), and then exits. This causes an additional span in the traces, so needed to create new snapshots.

Test coverage

This gives basic snapshot testing on 2022 which is the only version I could get to work given the hosted images on AzDo and the available python base images for the test agent. I think that is sufficient - we are going to have additional end-to-end testing of Windows images in system tests anyway.

Other details

Part of a stack of PRs:

datadog-ddstaging · 2025-02-10T12:16:23Z

Datadog Report

Branch report: andrew/win-ssi/3-smoke-tests
Commit report: a774929
Test service: dd-trace-dotnet

✅ 0 Failed, 245238 Passed, 2400 Skipped, 18h 33m 53.31s Total Time

andrewlock · 2025-02-10T13:48:41Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (69ms)  : 65, 74
     .   : milestone, 69,
    master - mean (69ms)  : 66, 72
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (992ms)  : 966, 1018
     .   : milestone, 992,
    master - mean (991ms)  : 967, 1016
     .   : milestone, 991,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (102ms)  : 101, 104
     .   : milestone, 102,
    master - mean (102ms)  : 100, 105
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (667ms)  : 650, 685
     .   : milestone, 667,
    master - mean (674ms)  : 653, 695
     .   : milestone, 674,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (89ms)  : 86, 92
     .   : milestone, 89,
    master - mean (89ms)  : 87, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (630ms)  : 609, 650
     .   : milestone, 630,
    master - mean (629ms)  : 614, 645
     .   : milestone, 629,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (191ms)  : 186, 195
     .   : milestone, 191,
    master - mean (191ms)  : 187, 195
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (1,105ms)  : 1077, 1134
     .   : milestone, 1105,
    master - mean (1,101ms)  : 1074, 1128
     .   : milestone, 1101,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (271ms)  : 268, 275
     .   : milestone, 271,
    master - mean (271ms)  : 265, 277
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (865ms)  : 835, 896
     .   : milestone, 865,
    master - mean (882ms)  : 827, 936
     .   : milestone, 882,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6645) - mean (263ms)  : 260, 267
     .   : milestone, 263,
    master - mean (285ms)  : 269, 302
     .   : milestone, 285,

    section CallTarget+Inlining+NGEN
    This PR (6645) - mean (849ms)  : 815, 883
     .   : milestone, 849,
    master - mean (874ms)  : 762, 987
     .   : milestone, 874,

andrewlock · 2025-02-10T14:23:51Z

Benchmarks Report for tracer 🐌

Benchmarks for #6645 compared to master:

1 benchmarks are faster, with geometric mean 1.205
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6645

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472	1.205	19,398.53	16,092.05	several?

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.46μs	48.3ns	345ns	0.0219	0.00876	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.6μs	59.5ns	385ns	0.0205	0.0103	0	5.8 KB
master	`StartStopWithChild`	net472	19.5μs	113ns	988ns	1.06	0.334	0.111	6.22 KB
#6645	`StartStopWithChild`	net6.0	7.91μs	45.2ns	336ns	0.016	0.00801	0	5.6 KB
#6645	`StartStopWithChild`	netcoreapp3.1	10.1μs	56.5ns	396ns	0.0198	0.00989	0	5.8 KB
#6645	`StartStopWithChild`	net472	16.1μs	42.9ns	166ns	1.04	0.311	0.0957	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	499μs	356ns	1.38μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	660μs	394ns	1.47μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	844μs	666ns	2.58μs	0.425	3.3 KB
#6645	`WriteAndFlushEnrichedTraces`	net6.0	488μs	113ns	422ns	0	2.7 KB
#6645	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	648μs	282ns	1.06μs	0	2.7 KB
#6645	`WriteAndFlushEnrichedTraces`	net472	842μs	782ns	3.03μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	128μs	325ns	1.26μs	0.189	14.47 KB
master	`SendRequest`	netcoreapp3.1	144μs	453ns	1.76μs	0.216	17.27 KB
master	`SendRequest`	net472	0ns	0ns	0ns	0	0 b
#6645	`SendRequest`	net6.0	131μs	379ns	1.47μs	0.196	14.47 KB
#6645	`SendRequest`	netcoreapp3.1	147μs	233ns	904ns	0.22	17.27 KB
#6645	`SendRequest`	net472	0.000111ns	7.7E‑05ns	0.000278ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	574μs	2.86μs	11.8μs	0.622	0	0	41.53 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	697μs	3.86μs	24.1μs	0.351	0	0	41.77 KB
master	`WriteAndFlushEnrichedTraces`	net472	846μs	2.99μs	11.2μs	8.28	2.48	0.414	53.3 KB
#6645	`WriteAndFlushEnrichedTraces`	net6.0	590μs	3.19μs	18.3μs	0.282	0	0	41.63 KB
#6645	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	660μs	3.59μs	21.2μs	0.363	0	0	41.86 KB
#6645	`WriteAndFlushEnrichedTraces`	net472	850μs	3.32μs	12.9μs	8.12	2.44	0.406	53.29 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.35μs	1.66ns	6.44ns	0.0142	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	1.91ns	6.89ns	0.0132	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.07μs	2.36ns	9.12ns	0.156	0.00104	987 B
#6645	`ExecuteNonQuery`	net6.0	1.36μs	1.27ns	4.92ns	0.0143	0	1.02 KB
#6645	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	2.02ns	7.82ns	0.013	0	1.02 KB
#6645	`ExecuteNonQuery`	net472	2.08μs	1.2ns	4.49ns	0.156	0.00103	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.19μs	0.646ns	2.42ns	0.0137	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.6μs	1.13ns	4.25ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.53μs	1.88ns	7.28ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.33μs	0.499ns	1.87ns	0.0133	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.65μs	1.13ns	4.24ns	0.0132	1.02 KB
master	`CallElasticsearchAsync`	net472	2.74μs	1.45ns	5.62ns	0.166	1.05 KB
#6645	`CallElasticsearch`	net6.0	1.3μs	0.433ns	1.68ns	0.0137	976 B
#6645	`CallElasticsearch`	netcoreapp3.1	1.52μs	1.1ns	4.25ns	0.0131	976 B
#6645	`CallElasticsearch`	net472	2.49μs	2.42ns	9.38ns	0.158	995 B
#6645	`CallElasticsearchAsync`	net6.0	1.26μs	0.567ns	2.2ns	0.0133	952 B
#6645	`CallElasticsearchAsync`	netcoreapp3.1	1.61μs	0.594ns	2.22ns	0.0137	1.02 KB
#6645	`CallElasticsearchAsync`	net472	2.6μs	0.941ns	3.26ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.43μs	0.645ns	2.41ns	0.0129	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.62μs	0.616ns	2.38ns	0.0122	952 B
master	`ExecuteAsync`	net472	1.78μs	0.3ns	1.12ns	0.145	915 B
#6645	`ExecuteAsync`	net6.0	1.3μs	0.357ns	1.38ns	0.0131	952 B
#6645	`ExecuteAsync`	netcoreapp3.1	1.62μs	0.755ns	2.82ns	0.0123	952 B
#6645	`ExecuteAsync`	net472	1.87μs	0.422ns	1.52ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.39μs	1.22ns	4.55ns	0.033	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.26μs	1.89ns	7.07ns	0.0393	2.85 KB
master	`SendAsync`	net472	7.42μs	1.71ns	6.4ns	0.493	3.12 KB
#6645	`SendAsync`	net6.0	4.36μs	1.89ns	7.08ns	0.0327	2.31 KB
#6645	`SendAsync`	netcoreapp3.1	5.37μs	2.14ns	8.3ns	0.0376	2.85 KB
#6645	`SendAsync`	net472	7.4μs	1.61ns	6.25ns	0.492	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.54μs	1.17ns	4.37ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.14μs	1.01ns	3.78ns	0.0226	1.64 KB
master	`EnrichedLog`	net472	2.55μs	0.754ns	2.82ns	0.249	1.57 KB
#6645	`EnrichedLog`	net6.0	1.47μs	1.06ns	3.97ns	0.0227	1.64 KB
#6645	`EnrichedLog`	netcoreapp3.1	2.16μs	1.91ns	7.13ns	0.0224	1.64 KB
#6645	`EnrichedLog`	net472	2.66μs	1.65ns	5.73ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	111μs	138ns	536ns	0.0557	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	116μs	127ns	493ns	0.0575	0	4.28 KB
master	`EnrichedLog`	net472	149μs	172ns	667ns	0.669	0.223	4.46 KB
#6645	`EnrichedLog`	net6.0	114μs	289ns	1.12μs	0.0576	0	4.28 KB
#6645	`EnrichedLog`	netcoreapp3.1	117μs	173ns	671ns	0.058	0	4.28 KB
#6645	`EnrichedLog`	net472	150μs	102ns	369ns	0.673	0.224	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.12μs	1.06ns	4.11ns	0.0311	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.16μs	1.22ns	4.58ns	0.0295	2.2 KB
master	`EnrichedLog`	net472	4.81μs	0.827ns	3.09ns	0.319	2.02 KB
#6645	`EnrichedLog`	net6.0	2.94μs	1.1ns	4.27ns	0.0309	2.2 KB
#6645	`EnrichedLog`	netcoreapp3.1	4.26μs	1.77ns	6.85ns	0.0297	2.2 KB
#6645	`EnrichedLog`	net472	4.95μs	0.984ns	3.81ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.44μs	1.9ns	7.36ns	0.0157	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.8μs	0.923ns	3.58ns	0.0154	1.14 KB
master	`SendReceive`	net472	2.15μs	1.93ns	7.49ns	0.183	1.16 KB
#6645	`SendReceive`	net6.0	1.37μs	1ns	3.87ns	0.0157	1.14 KB
#6645	`SendReceive`	netcoreapp3.1	1.78μs	3.34ns	12.9ns	0.0155	1.14 KB
#6645	`SendReceive`	net472	2.12μs	1.68ns	6.5ns	0.184	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.76μs	7.62ns	29.5ns	0.0219	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.87μs	2.24ns	8.67ns	0.0211	1.65 KB
master	`EnrichedLog`	net472	4.37μs	3.69ns	14.3ns	0.324	2.04 KB
#6645	`EnrichedLog`	net6.0	2.88μs	0.851ns	3.19ns	0.0217	1.6 KB
#6645	`EnrichedLog`	netcoreapp3.1	3.97μs	4.09ns	15.8ns	0.0218	1.65 KB
#6645	`EnrichedLog`	net472	4.23μs	4.38ns	17ns	0.324	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	421ns	1.09ns	4.23ns	0.00795	576 B
master	`StartFinishSpan`	netcoreapp3.1	619ns	1.66ns	6.2ns	0.00765	576 B
master	`StartFinishSpan`	net472	640ns	1.02ns	3.95ns	0.0916	578 B
master	`StartFinishScope`	net6.0	489ns	0.431ns	1.67ns	0.00979	696 B
master	`StartFinishScope`	netcoreapp3.1	741ns	2ns	7.76ns	0.00924	696 B
master	`StartFinishScope`	net472	952ns	2.84ns	11ns	0.104	658 B
#6645	`StartFinishSpan`	net6.0	417ns	0.537ns	2.08ns	0.00819	576 B
#6645	`StartFinishSpan`	netcoreapp3.1	563ns	1.03ns	3.98ns	0.00788	576 B
#6645	`StartFinishSpan`	net472	605ns	1.16ns	4.48ns	0.0918	578 B
#6645	`StartFinishScope`	net6.0	486ns	0.821ns	3.18ns	0.00971	696 B
#6645	`StartFinishScope`	netcoreapp3.1	703ns	0.613ns	2.29ns	0.0095	696 B
#6645	`StartFinishScope`	net472	873ns	1.22ns	4.73ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	660ns	1.49ns	5.76ns	0.00977	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	975ns	1.33ns	4.96ns	0.00904	696 B
master	`RunOnMethodBegin`	net472	1.17μs	3.56ns	13.3ns	0.104	658 B
#6645	`RunOnMethodBegin`	net6.0	704ns	0.52ns	2.01ns	0.00984	696 B
#6645	`RunOnMethodBegin`	netcoreapp3.1	892ns	3.26ns	12.6ns	0.00946	696 B
#6645	`RunOnMethodBegin`	net472	1.11μs	1.83ns	7.08ns	0.105	658 B

link04 · 2025-02-11T17:59:56Z

tracer/build/smoke_test_snapshots/smoke_test_iis_snapshots.json

+[
+  [
+    {
+      "trace_id": 4102954709290435042,


Just curious since I don't know, so for this one we don't need to scrub the ids nor any other attribute or use generic values?

Exactly - the dd-apm-test-agent takes care of that. IIRC I did scrub some of the values previously, just to make that clearer, but I was lazy here 🙈

andrewlock · 2025-02-12T00:49:35Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6645) (11.136M)   : 0, 11136180
    master (11.106M)   : 0, 11106032
    benchmarks/2.9.0 (11.045M)   : 0, 11045405

    section Automatic
    This PR (6645) (7.154M)   : 0, 7153983
    master (7.094M)   : 0, 7093671
    benchmarks/2.9.0 (7.885M)   : 0, 7885346

    section Trace stats
    master (7.448M)   : 0, 7448383

    section Manual
    master (10.999M)   : 0, 10999437

    section Manual + Automatic
    This PR (6645) (6.691M)   : 0, 6690851
    master (6.752M)   : 0, 6752044

    section DD_TRACE_ENABLED=0
    master (10.124M)   : 0, 10123660

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6645) (9.669M)   : 0, 9668903
    benchmarks/2.9.0 (9.586M)   : 0, 9586476

    section Automatic
    This PR (6645) (6.554M)   : 0, 6554377

    section Manual + Automatic
    This PR (6645) (5.916M)   : 0, 5916258

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6645) (10.206M)   : 0, 10205652

    section Automatic
    This PR (6645) (6.718M)   : 0, 6717962

    section Manual + Automatic
    This PR (6645) (6.276M)   : 0, 6275931

…6643) ## Summary of changes Creates an "installer" executable, designed to be executed by the fleet installer as part of configuring Windows SSI. ## Reason for change We don't want to use the existing MSI, primarily because it requires stopping and starting IIS when you update it, otherwise we risk causing crashes. The datadog-installer is responsible for copying the files added to the Windows OCI image, and running the Datadog.FleetInstaller.exe executable to configure the application. Similarly, it calls this exe when uninstalling a tracer version or removing the product ## Implementation details - Uses the fusion.dll API directly to manage the GAC entries, based on the generated PInvoke APIs defined [here](https://github.com/dotnet/pinvoke/tree/main/src/Fusion) - Sets the envivornment variables by setting the `environmentVariables` section in applicationHost.config. Uses [the _Microsoft.Web.Administration_ NuGet](https://www.nuget.org/packages/Microsoft.Web.Administration) to interact with the native API. ## Test coverage Manually tested. Subsequent PRs in the stack will add smoke and integration tests. ## Other details Explicitly designed to run on Windows Server 2016+, so targets .NET FX for simplicity. The _Microsoft.Web.Administration_ nuget is a .NET Standard 1.x dll, which creates an annoying number of dlls in the output, but it's just ugly, it works fine. There's a fair amount of duplication in the Fusion API/Microsoft.Web.Administration code with what the `dd-dotnet`/`dd-trace` tool currently does. We could certainly look at consolidating that down the line, and potentially rewriting the fleet installer tool to be a NativeAOT executable, but it's probably not worth the effort at this point. The explicit `<RuntimeIdentifier>win-x64</RuntimeIdentifier>` in the _.csproj_ is an odd one. Without it, for some reason, the gitlab build restores for x86 but tries to build for x64 which is... weird. Also, can't repro the issue locally. _Really_ don't understand what's happening there, but this works so... 🙈 Part of a stack of PRs: - #6643 👈 - #6644 - #6645

…6644) ## Summary of changes - Build the fleet installer exe in GitLab - Include the fleet installer exe in Windows OCI - Produce Windows OCI images in Staging ## Reason for change We need to build the fleet installer executable, and include it in a Windows OI image ## Implementation details - Add a stage `BuildFleetInstaller` to explicitly publish the fleet installer - Update the `download-single-step-artifacts` to grab the `windows-tracer-home.zip` and fleet installer artifact from the `Build` stage - Update the OCI image For tagged releases this flow is a bit more convoluted. We need to grab the fleet-instller.exe artifact from the s3 upload that's done in the `publish` step, because we don't store it on the GitHub release. It's a bit ugly, but it works. ## Test coverage This is the test - if it builds, and the OCI image contains the expected files, we're good ## Other details Note that _currently_, the final OCI images will _not_ contain Windows layers, until we unblock that both in our gitlab YAML, and in the libdatadog-build one pipeline. Part of a stack of PRs: - #6643 - #6644 👈 - #6645

This reverts commit 90eefb8eb8528788dd91dfa47701533e07b9c576.

This reverts commit dda17b3.

andrewlock added area:tests unit tests, integration tests area:test-apps apps used to test integrations area:ssi labels Feb 10, 2025

andrewlock requested review from a team as code owners February 10, 2025 11:46

andrewlock force-pushed the andrew/win-ssi/2-build branch from 0b6eda5 to d090638 Compare February 10, 2025 11:47

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch from 6ac72d2 to 2df6a66 Compare February 10, 2025 11:47

This was referenced Feb 10, 2025

Create an "installer" executable for use with the Datadog installer #6643

Merged

Build the FleetInstaller executable and include in Windows OCI image #6644

Merged

andrewlock force-pushed the andrew/win-ssi/2-build branch from d090638 to 0f5a72e Compare February 10, 2025 12:27

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch from 2df6a66 to d2623b6 Compare February 10, 2025 12:28

andrewlock force-pushed the andrew/win-ssi/2-build branch from 0f5a72e to 8f8c907 Compare February 10, 2025 12:44

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch from d2623b6 to 62db40f Compare February 10, 2025 12:44

andrewlock force-pushed the andrew/win-ssi/2-build branch from 8f8c907 to 91fc7fe Compare February 11, 2025 10:10

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch 2 times, most recently from 2116b2d to a774929 Compare February 11, 2025 15:44

link04 reviewed Feb 11, 2025

View reviewed changes

lucaspimentel approved these changes Feb 11, 2025

View reviewed changes

lucaspimentel requested a review from a team February 11, 2025 18:32

kevingosse approved these changes Feb 11, 2025

View reviewed changes

andrewlock force-pushed the andrew/win-ssi/2-build branch from 2341ef9 to e3f1d41 Compare February 12, 2025 12:38

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch from a774929 to b335f25 Compare February 12, 2025 12:38

andrewlock force-pushed the andrew/win-ssi/2-build branch from e3f1d41 to 4236975 Compare February 12, 2025 15:32

Base automatically changed from andrew/win-ssi/2-build to master February 12, 2025 18:45

andrewlock added 14 commits February 12, 2025 19:32

wip fleet installer smoke tests

c46fb37

More iteration

d99e9b6

TEMP bypass build disable retries (unnecessary)

dcc982d

Revert "TEMP bypass build disable retries (unnecessary)"

7dc4952

This reverts commit 90eefb8eb8528788dd91dfa47701533e07b9c576.

Add support for handling IIS hosting in the aspnetcore smoke test

24e9e7f

Add iis-specific snapshot

d439776

Try running --fail-with-body in Windows snapshots too

07f37a5

Clean up smoke tests a little

069942d

Fleet installer smoke tests should run on all PRs

374fec7

Nothing's ever easy, is it

a220bbd

Disable autoStart as it causes flake

f428309

Revert "Nothing's ever easy, is it"

6aa4715

This reverts commit dda17b3.

Only test 2022

cc071f4

Fix variable generation

51e2d0f

andrewlock force-pushed the andrew/win-ssi/3-smoke-tests branch from b335f25 to 51e2d0f Compare February 12, 2025 19:32

andrewlock merged commit f74c400 into master Feb 13, 2025
140 of 142 checks passed

andrewlock deleted the andrew/win-ssi/3-smoke-tests branch February 13, 2025 09:35

github-actions bot added this to the vNext-v3 milestone Feb 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add smoke tests for fleet installer #6645

Add smoke tests for fleet installer #6645

andrewlock commented Feb 10, 2025 •

edited

Loading

datadog-ddstaging bot commented Feb 10, 2025 •

edited

Loading

andrewlock commented Feb 10, 2025 •

edited

Loading

andrewlock commented Feb 10, 2025 •

edited

Loading

Faster 🎉 in #6645

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

link04 Feb 11, 2025

andrewlock Feb 12, 2025

andrewlock commented Feb 12, 2025

Add smoke tests for fleet installer #6645

Add smoke tests for fleet installer #6645

Conversation

andrewlock commented Feb 10, 2025 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Feb 10, 2025 • edited Loading

Datadog Report

andrewlock commented Feb 10, 2025 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Feb 10, 2025 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Faster 🎉 in #6645

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

link04 Feb 11, 2025

Choose a reason for hiding this comment

andrewlock Feb 12, 2025

Choose a reason for hiding this comment

andrewlock commented Feb 12, 2025

Throughput/Crank Report ⚡

andrewlock commented Feb 10, 2025 •

edited

Loading

datadog-ddstaging bot commented Feb 10, 2025 •

edited

Loading

andrewlock commented Feb 10, 2025 •

edited

Loading

andrewlock commented Feb 10, 2025 •

edited

Loading