Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add @aws-sdk/client-dynamodb instrumentation #3530

Merged
merged 40 commits into from
Sep 18, 2023
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
b5d4220
feat: add dynamodb v3 instrumentation
david-luna Jul 11, 2023
2323ba0
chore: add tests for dynamodb v3
david-luna Jul 11, 2023
086a810
chore: update comments
david-luna Jul 11, 2023
85bf88f
chore: update requires
david-luna Jul 11, 2023
9a2c0e4
chore: add missing tests
david-luna Jul 17, 2023
8504fbb
docs: update docs
david-luna Jul 17, 2023
0ec8f76
chore: update TAV
david-luna Jul 17, 2023
f89c76e
chore: update test scripts
david-luna Jul 17, 2023
fc310dc
Merge branch 'main' into dluna/2958-aws-dynamodb-v3-instr
david-luna Jul 17, 2023
b068570
chore: fix test.sh
david-luna Jul 17, 2023
2b9a826
chore: fix changelog
david-luna Jul 17, 2023
4735d0a
chore: hold on OTel attributes
david-luna Jul 17, 2023
7bc5d97
chore: add sample ouput gist
david-luna Jul 17, 2023
4457c15
chore: revert test.sh
david-luna Jul 17, 2023
fb886f4
chore: add script logs
david-luna Jul 17, 2023
e5d1142
chore: add script logs
david-luna Jul 17, 2023
2962d5e
chore: add script logs
david-luna Jul 17, 2023
e8446f5
chore: update dependencies
david-luna Jul 17, 2023
8c888dc
chore: aling lock file with main
david-luna Jul 17, 2023
d1c3fe0
chore: update lock file
david-luna Jul 17, 2023
66b423e
chore: merge main branch
david-luna Aug 1, 2023
cdd7637
chore: add lock dynamodb dependency
david-luna Aug 1, 2023
8240192
chore: lint fix dynamodb files
david-luna Aug 1, 2023
8933b46
chore: add ESM tests
david-luna Aug 1, 2023
2f296ec
chore: fix linting issues
david-luna Aug 1, 2023
34218cf
chore: remove OTel code
david-luna Aug 1, 2023
036b584
chore: update comments
david-luna Aug 1, 2023
fe33e08
chore: merge main branch
david-luna Sep 6, 2023
fad971f
chore: add dynamodb into tav.json
david-luna Sep 6, 2023
c0543b8
chore: update nede version check in tests
david-luna Sep 6, 2023
6dbd591
chore: fix lint
david-luna Sep 6, 2023
6329b53
chore: change iitm range computation
david-luna Sep 6, 2023
3843b41
tests: better range calculation
david-luna Sep 6, 2023
3dbac43
Merge branch 'main' into dluna/2958-aws-dynamodb-v3-instr
david-luna Sep 14, 2023
f917437
chore: update tav versions for S3 and dynamodb
david-luna Sep 14, 2023
21fc7cf
chore: remove logs
david-luna Sep 14, 2023
2eebdfa
Update test/instrumentation/modules/@aws-sdk/fixtures/use-client-dyna…
david-luna Sep 18, 2023
e0511cc
Update lib/instrumentation/modules/@aws-sdk/client-dynamodb.js
david-luna Sep 18, 2023
3b64c75
tests: add new IITM range constant
david-luna Sep 18, 2023
7879891
chore: lit fix
david-luna Sep 18, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci/scripts/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ elif [[ -n "${TAV_MODULE}" ]]; then
memcached)
DOCKER_COMPOSE_FILE=docker-compose-memcached.yml
;;
aws-sdk|@aws-sdk/client-s3)
aws-sdk|@aws-sdk/client-s3|@aws-sdk/client-dynamodb)
david-luna marked this conversation as resolved.
Show resolved Hide resolved
DOCKER_COMPOSE_FILE=docker-compose-localstack.yml
;;
*)
Expand Down
1 change: 1 addition & 0 deletions .ci/tav.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"modules": [
"@apollo/server",
"@aws-sdk/client-s3",
"@aws-sdk/client-dynamodb",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for reviewer: still under the limit and I guess the matrix will be reduced once we drop older node versions.

"@elastic/elasticsearch",
"@hapi/hapi",
"@opentelemetry/api",
Expand Down
21 changes: 15 additions & 6 deletions .tav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -512,13 +512,12 @@ aws-sdk:
- node test/instrumentation/modules/aws-sdk/sqs.test.js
- node test/instrumentation/modules/aws-sdk/dynamodb.test.js

# For all AWS-SDK clients want this version range:
# versions: '>=3.15.0 <4'
# However, @awk-sdk/client-* releases *very* frequently (almost every day) and there
# is no need to test *all* those releases. Instead we statically list a subset
# of versions to test.
'@aws-sdk/client-s3':
# We want this version range:
# versions: '>=3.15.0 <4'
# However, @awk-sdk/client-s3 releases *very* frequently (almost every day) and there
# is no need to test *all* those releases. Instead we statically list a subset
# of versions to test.
#
# Maintenance note: This should be updated periodically using:
# node ./dev-utils/tav-versions.js @aws-sdk/client-s3 ">=3.15.0 <4"
#
Expand All @@ -528,6 +527,16 @@ aws-sdk:
- node test/instrumentation/modules/@aws-sdk/client-s3.test.js
node: '>=14'

'@aws-sdk/client-dynamodb':
# Maintenance note: This should be updated periodically using:
# node ./dev-utils/tav-versions.js @aws-sdk/client-dynamodb ">=3.15.0 <4"
#
# Test v3.15.0, every N=41 of 210 releases, and current latest.
versions: '3.15.0 || 3.53.0 || 3.154.0 || 3.216.0 || 3.294.0 || 3.360.0 || 3.370.0 || >3.370.0 <4'
commands:
- node test/instrumentation/modules/@aws-sdk/client-s3.test.js
node: '>=14'

# - [email protected] added its diagnostics_channel support.
# - In [email protected] the `request.origin` property was added, which we need
# in the 'undici:request:create' diagnostic message.
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ Notes:
[float]
===== Features

* Add support for `@aws-sdk/client-dynamodb`, one of the AWS SDK v3 clients.
({pull}XXX[#XXXX])

[float]
===== Bug fixes

Expand Down
23 changes: 0 additions & 23 deletions dev-utils/aws-sdk-s3-client-tav-versions.sh

This file was deleted.

1 change: 1 addition & 0 deletions docs/supported-technologies.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ The Node.js agent will automatically instrument the following modules to give yo
|Module |Version |Note
|https://www.npmjs.com/package/aws-sdk[aws-sdk] |>=2.858.0 <3 |Will instrument SQS send/receive/delete messages, all S3 methods, all DynamoDB methods, and the SNS publish method
|https://www.npmjs.com/package/@aws-sdk/client-s3[@aws-sdk/client-s3] |>=3.15.0 <4 |Will instrument all S3 methods
|https://www.npmjs.com/package/@aws-sdk/client-dynamodb[@aws-sdk/client-dynamodb] |>=3.15.0 <4 |Will instrument all DynamoDB methods
|https://www.npmjs.com/package/cassandra-driver[cassandra-driver] |>=3.0.0 <5 |Will instrument all queries
|https://www.npmjs.com/package/elasticsearch[elasticsearch] |>=8.0.0 |Will instrument all queries
|https://www.npmjs.com/package/@elastic/elasticsearch[@elastic/elasticsearch] |>=7.0.0 <9.0.0 |Will instrument all queries
Expand Down
151 changes: 151 additions & 0 deletions lib/instrumentation/modules/@aws-sdk/client-dynamodb.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
/*
* Copyright Elasticsearch B.V. and other contributors where applicable.
* Licensed under the BSD 2-Clause License; you may not use this file except in
* compliance with the BSD 2-Clause License.
*/

'use strict';

const constants = require('../../../constants');
const NAME = 'DynamoDB';
const TYPE = 'db';
const SUBTYPE = 'dynamodb';
const elasticAPMStash = Symbol('elasticAPMStash');

/**
* Returns middlewares to instrument an S3Client instance
*
* @param {import('@aws-sdk/client-dynamodb').DynamoDBClient} client
* @param {any} agent
* @returns {import('./smithy-client').AWSMiddlewareEntry[]}
david-luna marked this conversation as resolved.
Show resolved Hide resolved
*/
function dynamoDBMiddlewareFactory(client, agent) {
return [
{
middleware: (next, context) => async (args) => {
// Ensure there is a span from the wrapped `client.send()`.
const span = agent._instrumentation.currSpan();
if (!span || !(span.type === TYPE && span.subtype === SUBTYPE)) {
return await next(args);
}

const input = args.input;
const table = input && input.TableName;
// The given span comes with the operation name and we need to
// add the table if applies
if (table) {
span.name += ' ' + table;
}

let err;
let result;
let response;
let statusCode;
try {
result = await next(args);
response = result && result.response;
statusCode = response && response.statusCode;
} catch (ex) {
// Save the error for use in `finally` below, but re-throw it to
// not impact code flow.
err = ex;

// This code path happens with a GetObject conditional request
// that returns a 304 Not Modified.
statusCode = err && err.$metadata && err.$metadata.httpStatusCode;
throw ex;
} finally {
if (statusCode) {
span._setOutcomeFromHttpStatusCode(statusCode);
} else {
span._setOutcomeFromErrorCapture(constants.OUTCOME_FAILURE);
}
if (err && (!statusCode || statusCode >= 400)) {
agent.captureError(err, { skipOutcome: true });
}

// TODO: something related to region?
const config = client.config;
const region = await config.region();

// Set the db context
const dbContext = { type: SUBTYPE }; // dynamodb
if (region) {
dbContext.instance = region;
}
if (input && input.KeyConditionExpression) {
dbContext.statement = input.KeyConditionExpression;
}
span.setDbContext(dbContext);

// Set destination context
const destContext = {};
if (context[elasticAPMStash]) {
destContext.address = context[elasticAPMStash].hostname;
destContext.port = context[elasticAPMStash].port;
}
if (region) {
destContext.service = { resource: `dynamodb/${region}` };
destContext.cloud = { region };
}
span._setDestinationContext(destContext);

// TODO: review spec and add OTel attributes
// https://github.com/open-telemetry/opentelemetry-specification/blob/v1.20.0/semantic_conventions/trace/instrumentation/aws-sdk.yml#L435
// OTel attributes
// const otelAttrs = span._getOTelAttributes()

// otelAttrs['aws.dynamodb.system'] = 'dynamodb'
// otelAttrs['aws.dynamodb.operation'] = span.action
// if (table) {
// otelAttrs['aws.dynamodb.table_names'] = table
// }
// if (input && input.Select) {
// otelAttrs['aws.dynamodb.select'] = input.Select
// }
// if (input && input.ProjectionExpression) {
// otelAttrs['aws.dynamodb.projection'] = input.ProjectionExpression
// }
// // TODO: check result for consumed_capacity
// if (result && result.ConsumedCapacity) {
// otelAttrs['aws.dynamodb.consumed_capacity'] = JSON.stringify(result.ConsumedCapacity)
// }

span.end();
}

return result;
},
options: { step: 'initialize', priority: 'high', name: 'elasticAPMSpan' },
},
{
middleware: (next, context) => async (args) => {
trentm marked this conversation as resolved.
Show resolved Hide resolved
const req = args.request;
let port = req.port;

// Resolve port for HTTP(S) protocols
if (port === undefined) {
if (req.protocol === 'https:') {
port = 443;
} else if (req.protocol === 'http:') {
port = 80;
}
}

context[elasticAPMStash] = {
hostname: req.hostname,
port,
};
return next(args);
},
options: { step: 'finalizeRequest', name: 'elasticAPMHTTPInfo' },
},
];
}

module.exports = {
DYNAMODB_NAME: NAME,
DYNAMODB_TYPE: TYPE,
DYNAMODB_SUBTYPE: SUBTYPE,
dynamoDBMiddlewareFactory,
};
12 changes: 12 additions & 0 deletions lib/instrumentation/modules/@smithy/smithy-client.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ const {
S3_SUBTYPE,
s3MiddlewareFactory,
} = require('../@aws-sdk/client-s3');
const {
DYNAMODB_NAME,
DYNAMODB_TYPE,
DYNAMODB_SUBTYPE,
dynamoDBMiddlewareFactory,
} = require('../@aws-sdk/client-dynamodb');

/**
* We do alias them to a local type
Expand Down Expand Up @@ -56,6 +62,12 @@ const clientsConfig = {
SUBTYPE: S3_SUBTYPE,
factory: s3MiddlewareFactory,
},
DynamoDBClient: {
NAME: DYNAMODB_NAME,
TYPE: DYNAMODB_TYPE,
SUBTYPE: DYNAMODB_SUBTYPE,
factory: dynamoDBMiddlewareFactory,
},
};

module.exports = function (mod, agent, { name, version, enabled }) {
Expand Down
Loading
Loading