Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tracer] Add s3 instrumentation #6590

Open
wants to merge 59 commits into
base: master
Choose a base branch
from
+8,616 −1,322
Open

[tracer] Add s3 instrumentation #6590

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
abb2bae
add basic S3 instrumentation for `PutObject` and `PutObjectAsync`
nhulston Jan 27, 2025
99d4528
implement tags
nhulston Jan 27, 2025
1c6707a
create basic s3 sample app
nhulston Jan 27, 2025
01734d3
implement basic s3 integration tests
nhulston Jan 27, 2025
8e22164
run GeneratePackageVersions
nhulston Jan 27, 2025
1e3ee72
Add `aws.bucket.name` and `aws.object.key` tags
nhulston Jan 28, 2025
96dde22
Instrument `PutObject` and `PutObjectAsync`; set tags for bucket name…
nhulston Jan 28, 2025
f1cde9c
create basic instrumentation for CopyObject, CopyObjectAsync, Complet…
nhulston Jan 28, 2025
2085dfb
Run `GeneratePackageVersions` to update call targets
nhulston Jan 28, 2025
d3bb087
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Jan 28, 2025
205388b
update calltargets
nhulston Jan 28, 2025
308d567
Instrument CopyObject and CompleteMultipartUpload
nhulston Jan 28, 2025
387122d
create GetObject integration
nhulston Jan 28, 2025
4bffcbc
create DeleteObject integration
nhulston Jan 28, 2025
c9eb8e2
create ListObjects integration
nhulston Jan 28, 2025
0e4c0f0
create PutBucket integration
nhulston Jan 28, 2025
a6d606f
create DeleteBucket integration
nhulston Jan 28, 2025
239409f
refactor async integrations
nhulston Jan 28, 2025
9273310
create ListObjectsV2 integration
nhulston Jan 28, 2025
5e415a7
refactor integrations; remove ListObjectsV1 integration in favor of v2
nhulston Jan 28, 2025
675d9e9
run GeneratePackageVersions
nhulston Jan 28, 2025
57d7e35
create ListBuckets integration
nhulston Jan 28, 2025
dca6ed4
simplify comments
nhulston Jan 28, 2025
ed525ff
create DeleteObjects integration
nhulston Jan 28, 2025
6493d30
create InitiateMultipartUpload integration
nhulston Jan 28, 2025
bf0a5e1
create UploadPart integration
nhulston Jan 28, 2025
0736553
fix ListBucketsAsync file name
nhulston Jan 28, 2025
dd6c0a2
run GeneratePackageVersions
nhulston Jan 28, 2025
49beaa1
fix span kind to Client; make `spanKind` parameter have default value…
nhulston Jan 29, 2025
bf5e58b
add AwsS3Common unit tests
nhulston Jan 29, 2025
7fcd5b2
update SpanMetadataRules tests
nhulston Jan 29, 2025
00a8dc0
implement sample app
nhulston Jan 29, 2025
ccfc372
fix s3 launch settings
nhulston Jan 29, 2025
21d730f
update tags
nhulston Jan 30, 2025
1d8957d
fix integration tests (no uppercase bucket names)
nhulston Jan 30, 2025
33135ba
fix unit tests
nhulston Jan 30, 2025
fcda3fa
update v0 operation name
nhulston Jan 30, 2025
32b8589
add NetCore snapshots
nhulston Jan 30, 2025
4163a58
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Jan 30, 2025
65c52bd
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Jan 31, 2025
7d631d0
update snapshots
nhulston Jan 31, 2025
f3cb287
manually construct s3 request paths
nhulston Jan 31, 2025
b2b2f2d
add test coverage for every instrumented operation
nhulston Jan 31, 2025
dc83a86
update rules and snapshots. Operations like `ListBuckets` obviously w…
nhulston Jan 31, 2025
6fca491
fix localstack checksum error in ci. https://github.com/aws/aws-sdk-n…
nhulston Jan 31, 2025
340942b
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Feb 3, 2025
7e6ad26
Add .NET Framework snapshots
bouwkast Feb 5, 2025
98ddff9
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Feb 5, 2025
02e2bad
Add regex scrubber for http.url tag in integration tests; revert cust…
nhulston Feb 5, 2025
d14d9c0
revert RUntimePipelineInvokeIntegration files
nhulston Feb 5, 2025
621491c
Add `DeleteObjectsAsync` and `GetObjectAsync` to integration tests
nhulston Feb 5, 2025
2dace21
Update NetCore snapshots
nhulston Feb 5, 2025
c7f486a
Test expected span count
nhulston Feb 5, 2025
00b8041
Update .NET Framework snapshots
bouwkast Feb 6, 2025
3a2d0f8
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Feb 6, 2025
60aa1ba
run GeneratePackageVersions
nhulston Feb 6, 2025
f0cff60
Bump minimum version to 3.3.0 and generate .NET Framework targets
bouwkast Feb 7, 2025
7923548
Merge branch 'master' into nicholas.hulston/add-s3-instrumentation
nhulston Feb 7, 2025
51e9d22
run GeneratePackageVersions
nhulston Feb 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Datadog.Trace.OSX.slnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
"tracer\\test\\test-applications\\integrations\\Samples.AWS.EventBridge\\Samples.AWS.EventBridge.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AWS.Kinesis\\Samples.AWS.Kinesis.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AWS.Lambda\\Samples.AWS.Lambda.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AWS.S3\\Samples.AWS.S3.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AWS.SimpleNotificationService\\Samples.AWS.SimpleNotificationService.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AWS.SQS\\Samples.AWS.SQS.csproj",
"tracer\\test\\test-applications\\integrations\\Samples.AzureServiceBus\\Samples.AzureServiceBus.csproj",
Expand Down
7 changes: 7 additions & 0 deletions Datadog.Trace.Samples.g.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.AzureFunctions.V4Is
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.GoogleProtobuf", "tracer\test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj", "{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.AWS.S3", "tracer\test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj", "{0C0578CB-3B67-4F95-8547-206CD2A560CD}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -1045,6 +1047,10 @@ Global
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Release|Any CPU.Build.0 = Release|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(NestedProjects) = preSolution
{9518425A-36A5-4B8F-B0B8-6137DB88441D} = {8CEC2042-F11C-49F5-A674-2355793B600A}
Expand Down Expand Up @@ -1218,5 +1224,6 @@ Global
{5D2C6B9C-FCE2-4E46-B4ED-BC3B11CFBB3C} = {C4C1E313-C7C1-4490-AECE-0DD0062380A4}
{0F8EAB52-0C5B-4F60-92C5-42FAC21F4E77} = {C4C1E313-C7C1-4490-AECE-0DD0062380A4}
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9} = {BAF8F246-3645-42AD-B1D0-0F7EAFBAB34A}
{0C0578CB-3B67-4F95-8547-206CD2A560CD} = {BAF8F246-3645-42AD-B1D0-0F7EAFBAB34A}
EndGlobalSection
EndGlobal
8 changes: 8 additions & 0 deletions Datadog.Trace.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.AzureFunctions.V4Is
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.GoogleProtobuf", "tracer\test\test-applications\integrations\Samples.GoogleProtobuf\Samples.GoogleProtobuf.csproj", "{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.AWS.S3", "tracer\test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj", "{0C0578CB-3B67-4F95-8547-206CD2A560CD}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -1453,6 +1455,10 @@ Global
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Release|Any CPU.ActiveCfg = Release|Any CPU
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9}.Release|Any CPU.Build.0 = Release|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0C0578CB-3B67-4F95-8547-206CD2A560CD}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -1689,6 +1695,7 @@ Global
{5D2C6B9C-FCE2-4E46-B4ED-BC3B11CFBB3C} = {C4C1E313-C7C1-4490-AECE-0DD0062380A4}
{0F8EAB52-0C5B-4F60-92C5-42FAC21F4E77} = {C4C1E313-C7C1-4490-AECE-0DD0062380A4}
{EF8C4CCE-E79C-4D78-BF31-222A11E198B9} = {BAF8F246-3645-42AD-B1D0-0F7EAFBAB34A}
{0C0578CB-3B67-4F95-8547-206CD2A560CD} = {BAF8F246-3645-42AD-B1D0-0F7EAFBAB34A}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {160A1D00-1F5B-40F8-A155-621B4459D78F}
Expand All @@ -1698,6 +1705,7 @@ Global
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{061ab58b-8235-4dae-8d56-5f081dd78f5e}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{086ff8a0-9cee-470a-9751-78b0f1340649}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{0980bcdd-a231-42d1-b689-41a41bba161a}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{0c0578cb-3b67-4f95-8547-206cd2a560cd}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{0d996eee-7c04-4888-af48-9c1e2f261a00}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{0df4363a-0df4-4882-a39f-3c9f404b8de5}*SharedItemsImports = 5
tracer\test\test-applications\Samples.Shared\Samples.Shared.projitems*{0e036453-2c80-4fc9-a517-771f0071734b}*SharedItemsImports = 5
Expand Down
6 changes: 3 additions & 3 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ services:
localstack_arm64:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb,events
- SERVICES=sns,sqs,kinesis,dynamodb,events,s3
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down Expand Up @@ -92,7 +92,7 @@ services:
localstack:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb,events
- SERVICES=sns,sqs,kinesis,dynamodb,events,s3
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down Expand Up @@ -1077,7 +1077,7 @@ services:
localstack_osx_arm64:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb,events
- SERVICES=sns,sqs,kinesis,dynamodb,events,s3
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down
18 changes: 18 additions & 0 deletions tracer/build/PackageVersionsGeneratorDefinitions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,24 @@
}
]
},
{
"IntegrationName": "AwsS3",
"SampleProjectName": "Samples.AWS.S3",
"NugetPackageSearchName": "AWSSDK.S3",
"MinVersion": "3.3.0",
"MaxVersionExclusive": "4.0.0",
"SpecificVersions": [
"3.3.*",
"3.5.*",
"3.7.*"
],
"VersionConditions": [
{
"MaxVersionExclusive": "3.3.0",
"IncludeOnlyTargetFrameworks": ["net462"]
}
]
},
{
"IntegrationName": "MongoDB",
"SampleProjectName": "Samples.MongoDB",
Expand Down
54 changes: 54 additions & 0 deletions tracer/build/PackageVersionsLatestMajors.g.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,60 @@ NOTE: This code was generated by the GeneratePackageVersions tool. To safely
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net462</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net5.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net6.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net7.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net8.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.S3\Samples.AWS.S3.csproj">
<Properties>ApiVersion=3.7.414;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net9.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.S3</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.MongoDB\Samples.MongoDB.csproj">
<Properties>ApiVersion=2.30.0;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net462</TargetFramework>
Expand Down
Loading
Loading