Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aww-eventbridge-sqs): add a dlq for the event rule #1253

Merged
merged 12 commits into from
Jan 31, 2025
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,9 @@ constructStack.getEncryptionKey().addToResourcePolicy(policyStatement);
|existingEventBusInterface?|[`events.IEventBus`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.IEventBus.html)| Optional user-provided custom EventBus for construct to use. Providing both this and `eventBusProps` results an error.|
|eventBusProps?|[`events.EventBusProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.EventBusProps.html)|Optional user-provided properties to override the default properties when creating a custom EventBus. Setting this value to `{}` will create a custom EventBus using all default properties. If neither this nor `existingEventBusInterface` is provided the construct will use the `default` EventBus. Providing both this and `existingEventBusInterface` results an error.|
|eventRuleProps|[`events.RuleProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.RuleProps.html)|User provided eventRuleProps to override the defaults. |
|targetProps?|[`eventtargets.SqsQueueProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events_targets.SqsQueueProps.html)|Optional user provided properties to define the SQS target on the Event Rule. If you specify a deadLetterQueue for the rule here, you are responsible for adding a resource polic. to the queue allowing events.amazonaws.com permission to SendMessage, GetQueueUrl and GetQueueAttributes. You cannot send a DLQ in this property and set deployEventRuleDlq to true. Default is undefined and all system defaults are used.|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: There's a typo here in the middle of 118, should be:

...responsible for adding a resource policy to the queue allowing...

|eventRuleDlqKeyProps|[kms.KeyProps](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.KeyProps.html)|Optional properties to define the key created to protect the ruleDlq. Only valid if deployRuleDlq is set to true. Defaults to CloudFormation defaults.|
| deployEventRuleDlq?|boolean|Whether to deploy a DLQ for the Event Rule. If set to `true`, this DLQ will receive any messages that can't be delivered to the target SQS queue. Defaults to `false`.|
|existingQueueObj?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|An optional, existing SQS queue to be used instead of the default queue. Providing both this and `queueProps` will cause an error.|
|queueProps?|[`sqs.QueueProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.QueueProps.html)|User provided props to override the default props for the SQS Queue. |
|enableQueuePurging?|`boolean`|Whether to grant additional permissions to the Lambda function enabling it to purge the SQS queue. Defaults to `false`.|
Expand All @@ -131,6 +134,8 @@ constructStack.getEncryptionKey().addToResourcePolicy(policyStatement);
|:-------------|:----------------|-----------------|
|eventBus?|[`events.IEventBus`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.IEventBus.html)|Returns the instance of events.IEventBus used by the construct|
|eventsRule|[`events.Rule`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.Rule.html)|Returns an instance of events.Rule created by the construct|
|eventRuleDlq?|`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|If the client sets deployEventRuleDlq to 'true', then this value will contain the DLQ set up for the rule.|
|eventRuleDlqKey|[kms.IKey](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.IKey.html)|The key created to encrypt the eventRuleDlq.|
|sqsQueue|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|Returns an instance of sqs.Queue created by the construct|
|encryptionKey?|[`kms.Key`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_kms.Key.html)|Returns an instance of kms Key used for the SQS queue.|
|deadLetterQueue?|[`sqs.Queue`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_sqs.Queue.html)|Returns an instance of the dead-letter SQS queue created by the pattern.|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as events from 'aws-cdk-lib/aws-events';
import * as eventtargets from 'aws-cdk-lib/aws-events-targets';
import * as kms from 'aws-cdk-lib/aws-kms';
import * as defaults from '@aws-solutions-constructs/core';
import { ServicePrincipal } from 'aws-cdk-lib/aws-iam';
Expand Down Expand Up @@ -42,6 +43,20 @@ export interface EventbridgeToSqsProps {
* @default - None
*/
readonly eventRuleProps: events.RuleProps;
/**
* Whether to deploy a DLQ for the Event Rule. If set to `true`, this DLQ will
* receive any messages that can't be delivered to the target SQS queue.
*
* @default - false
*/
readonly deployEventRuleDlq?: boolean;
/**
* Properties to define the key created to protect the ruleDlq
* Only valid if deployRuleDlq is set to true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on L55 has the old deployRuleDlq, should be deployEventRuleDlq

*
* @default - default props are used
*/
readonly eventRuleDlqKeyProps?: kms.KeyProps;
/**
* Existing instance of SQS queue object, providing both this and queueProps will cause an error.
*
Expand All @@ -60,6 +75,16 @@ export interface EventbridgeToSqsProps {
* @default - "false", disabled by default.
*/
readonly enableQueuePurging?: boolean;
/**
* Optional user provided properties to define the SQS target on the Event Rule
*
* If you specify a deadLetterQueue for the rule here, you are responsible for adding a resource policy
* to the queue allowing events.amazonaws.com permission to SendMessage, GetQueueUrl and GetQueueAttributes. You
* cannot send a DLQ in this property and set deployRuleDlq to true
*
* @default - undefined (all default values are used)
*/
readonly targetProps?: eventtargets.SqsQueueProps;
/**
* Optional user provided properties for the dead letter queue
*
Expand Down Expand Up @@ -105,6 +130,8 @@ export class EventbridgeToSqs extends Construct {
public readonly eventBus?: events.IEventBus;
public readonly eventsRule: events.Rule;
public readonly encryptionKey?: kms.IKey;
public readonly eventRuleDlq?: sqs.Queue;
public readonly eventRuleDlqKey?: kms.IKey;

/**
* @summary Constructs a new instance of the EventbridgeToSqs class.
Expand All @@ -118,6 +145,14 @@ export class EventbridgeToSqs extends Construct {
super(scope, id);
defaults.CheckSqsProps(props);
defaults.CheckEventBridgeProps(props);
// SqsQueueProps does not implement any common interface, so is unique to this construct,
// so we will check it here rather than in core
if ((props.targetProps?.deadLetterQueue) && (props.deployEventRuleDlq)) {
throw new Error('Cannot specify both targetProps.deadLetterQueue and deployDeadLetterQueue == true\n');
}
if (props.eventRuleDlqKeyProps && !props.deployEventRuleDlq) {
throw new Error('Cannot specify eventRuleDlqKeyProps without setting deployEventRuleDlq=true\n');
}

let enableEncryptionParam = props.enableEncryptionWithCustomerManagedKey;
if (props.enableEncryptionWithCustomerManagedKey === undefined ||
Expand All @@ -140,12 +175,28 @@ export class EventbridgeToSqs extends Construct {
this.encryptionKey = buildQueueResponse.key;
this.deadLetterQueue = buildQueueResponse.dlq;

const sqsEventTarget: events.IRuleTarget = {
bind: () => ({
id: this.sqsQueue.queueName,
arn: this.sqsQueue.queueArn
})
};
let constructEventTargetProps: eventtargets.SqsQueueProps = {};

if (defaults.CheckBooleanWithDefault(props.deployEventRuleDlq, false)) {

const buildRuleDlqResponse = defaults.buildQueue(this, 'ruleDlq', {
deployDeadLetterQueue: false,
enableEncryptionWithCustomerManagedKey: enableEncryptionParam,
encryptionKeyProps: props.eventRuleDlqKeyProps
});

this.eventRuleDlq = buildRuleDlqResponse.queue;
const ruleDlqKey = buildRuleDlqResponse.key;
ruleDlqKey?.grantEncryptDecrypt(new ServicePrincipal('events.amazonaws.com'));
this.eventRuleDlqKey = ruleDlqKey;

// TODO: Update docs on encrpytWithCustomerMasterKey
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking, did we want to keep this TODO in here for future or were we supposed to action on this as part of this PR?


constructEventTargetProps = defaults.consolidateProps(constructEventTargetProps, { deadLetterQueue: this.eventRuleDlq });
}

const sqsEventTargetProps = defaults.consolidateProps({}, props.targetProps, constructEventTargetProps);
const sqsEventTarget = new eventtargets.SqsQueue(this.sqsQueue, sqsEventTargetProps);

// build an event bus if existingEventBus is provided or eventBusProps are provided
this.eventBus = defaults.buildEventBus(this, {
Expand All @@ -163,7 +214,5 @@ export class EventbridgeToSqs extends Construct {
this.sqsQueue.grantPurge(new ServicePrincipal('events.amazonaws.com'));
}

// Policy for event to be able to send messages to the queue and Grant Event Bridge service access to the SQS queue encryption key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's allowing events to be sent to the queue now without this in there anymore? (Mostly curious but wanted to point it out just in case too)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a new aws-event-targets library since we first wrote this, the SqsQueue target object adds the permissions automatically. It doesn't add Purge however.

this.sqsQueue.grantSendMessages(new ServicePrincipal('events.amazonaws.com'));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import * as cdk from 'aws-cdk-lib';
import { EventbridgeToSqs, EventbridgeToSqsProps } from '../lib';
import * as events from "aws-cdk-lib/aws-events";
import * as sqs from "aws-cdk-lib/aws-sqs";
import { Template } from 'aws-cdk-lib/assertions';
import { Match, Template } from 'aws-cdk-lib/assertions';
import * as defaults from '@aws-solutions-constructs/core';

function deployNewStack(stack: cdk.Stack) {
Expand Down Expand Up @@ -495,7 +495,7 @@ test('Queue purging flag grants correct permissions', () => {
},
{
Action: [
"sqs:PurgeQueue",
"sqs:SendMessage",
"sqs:GetQueueAttributes",
"sqs:GetQueueUrl"
],
Expand All @@ -512,7 +512,7 @@ test('Queue purging flag grants correct permissions', () => {
},
{
Action: [
"sqs:SendMessage",
"sqs:PurgeQueue",
"sqs:GetQueueAttributes",
"sqs:GetQueueUrl"
],
Expand Down Expand Up @@ -559,3 +559,150 @@ test('check that CheckSqsProps is being called', () => {
};
expect(app).toThrowError("Error - Either provide queueProps or existingQueueObj, but not both.\n");
});

test('Check that rule dlq is not created by default', () => {
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
}
};
const testConstruct = new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
expect(testConstruct.eventRuleDlq).toBeUndefined();
const template = Template.fromStack(stack);
template.resourceCountIs("AWS::SQS::Queue", 2);
template.hasResourceProperties("AWS::Events::Rule", {
Targets: [
{
Id: "Target0",
DeadLetterConfig: Match.absent(),
}
]
});
});

test('Check that rule dlq is created when requested', () => {
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
deployEventRuleDlq: true
};
const testConstruct = new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
expect(testConstruct.eventRuleDlq).toBeDefined();
const template = Template.fromStack(stack);
template.resourceCountIs("AWS::SQS::Queue", 3);
template.hasResourceProperties("AWS::Events::Rule", {
Targets: [
{
Id: "Target0",
DeadLetterConfig: {
Arn: {
"Fn::GetAtt": [
Match.stringLikeRegexp("testeventbridgesqsruleDlq.*"),
"Arn"
]
}
},
}
]
});
});

test('Check that rule dlq is created when requested', () => {
const testMaxAge = cdk.Duration.seconds(47);
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
targetProps: {
maxEventAge: testMaxAge
}
};
new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
const template = Template.fromStack(stack);

template.hasResourceProperties("AWS::Events::Rule", {
Targets: [
{
Id: "Target0",
RetryPolicy: {
MaximumEventAgeInSeconds: 47
}
}
]
});
});

test('Check that client cannot submit their own Rule DLQ and ask for a DLQ to be created', () => {
// This may be enabled in the future, but first release
// is minimum viable product and takes full ownership of the Rule DLQ
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
deployEventRuleDlq: true,
targetProps: {
deadLetterQueue: {} as sqs.Queue
}
};
const app = () => {
new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
};
expect(app).toThrowError('Cannot specify both targetProps.deadLetterQueue and deployDeadLetterQueue == true\n');
});

test('Test that the construct uses provided eventRuleDlgKey properties', () => {
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
eventRuleDlqKeyProps: {
alias: 'test-alias'
},
deployEventRuleDlq: true,
};
new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
const template = Template.fromStack(stack);
template.hasResourceProperties("AWS::KMS::Alias", {
AliasName: "alias/test-alias",
TargetKeyId: {
"Fn::GetAtt": [
Match.stringLikeRegexp("testeventbridgesqsruleDlqKey.*"),
"Arn"
]
}
});
});

test('Thest that the eventRuleDlqKey is exposed as a property', () => {
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
deployEventRuleDlq: true,
};
const testConstruct = new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
expect(testConstruct.eventRuleDlqKey).toBeDefined();
});

test('Test that an error is thrown when eventRuleDlqKeyProps are provided but deployEventRuleDlq is not true', () => {
const stack = new cdk.Stack();
const props: EventbridgeToSqsProps = {
eventRuleProps: {
schedule: events.Schedule.rate(cdk.Duration.minutes(5))
},
eventRuleDlqKeyProps: {
alias: 'test-alias'
},
};
const app = () => {
new EventbridgeToSqs(stack, 'test-eventbridge-sqs', props);
};
expect(app).toThrowError('Cannot specify eventRuleDlqKeyProps without setting deployEventRuleDlq=true\n');
});
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"version":"39.0.0"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"version": "39.0.0",
"files": {
"8221310775997696fa75dec2375830f8d37a0d9d424ea6d55a03f453d4bec94c": {
"source": {
"path": "evtsqs-custom-target.template.json",
"packaging": "file"
},
"destinations": {
"current_account-current_region": {
"bucketName": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}",
"objectKey": "8221310775997696fa75dec2375830f8d37a0d9d424ea6d55a03f453d4bec94c.json",
"assumeRoleArn": "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cdk-hnb659fds-file-publishing-role-${AWS::AccountId}-${AWS::Region}"
}
}
}
},
"dockerImages": {}
}
Loading