Skip to content

Latest commit

 

History

History
963 lines (606 loc) · 37.1 KB

API.md

File metadata and controls

963 lines (606 loc) · 37.1 KB
Raw

API Reference

Constructs

SitemapFreshenerConstruct

Sitemap Freshener Construct.

This construct creates a Lambda function that reads from a Kinesis stream and writes sitemap files to an S3 bucket from the contents in the DynamoDB Table.

The Lambda function can be invoked with a start message or it can read the start message from the Kinesis stream.

The Lambda will write a freshenFile message back to the Kinesis stream for each sitemap file that exists for the specified type.

Changes toremove to removed and towrite to written in the DB for items in those states.

When repair mode is off the XML files are only written, not read.

Initializers 

import { SitemapFreshenerConstruct } from '@shutterstock/sitemaps-cdk'

new SitemapFreshenerConstruct(scope: Construct, id: string, props?: SitemapFreshnerConstructProps)
Name Type Description
scope constructs.Construct No description.
id string No description.
props SitemapFreshnerConstructProps No description.
scopeRequired
  • Type: constructs.Construct
idRequired
  • Type: string
propsOptional

Methods

Name Description
toString Returns a string representation of this construct.
toString 
public toString(): string

Returns a string representation of this construct.

Static Functions

Name Description
isConstruct Checks if x is a construct.
isConstruct 
import { SitemapFreshenerConstruct } from '@shutterstock/sitemaps-cdk'

SitemapFreshenerConstruct.isConstruct(x: any)

Checks if x is a construct.

Use this method instead of instanceof to properly detect Construct instances, even when the construct library is symlinked.

Explanation: in JavaScript, multiple copies of the constructs library on disk are seen as independent, completely different libraries. As a consequence, the class Construct in each copy of the constructs library is seen as a different class, and an instance of one class will not test as instanceof the other class. npm install will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the constructs library can be accidentally installed, and instanceof will behave unpredictably. It is safest to avoid using instanceof, and using this type-testing method instead.

xRequired
  • Type: any

Any object.

Properties

Name Type Description
node constructs.Node The tree node.
sitemapFreshenerLambdaFunction aws-cdk-lib.aws_lambda.Function No description.
nodeRequired 
public readonly node: Node;
  • Type: constructs.Node

The tree node.

sitemapFreshenerLambdaFunctionRequired 
public readonly sitemapFreshenerLambdaFunction: Function;
  • Type: aws-cdk-lib.aws_lambda.Function

SitemapsConstruct

Initializers 

import { SitemapsConstruct } from '@shutterstock/sitemaps-cdk'

new SitemapsConstruct(scope: Construct, id: string, props?: SitemapsConstructProps)
Name Type Description
scope constructs.Construct No description.
id string No description.
props SitemapsConstructProps No description.
scopeRequired
  • Type: constructs.Construct
idRequired
  • Type: string
propsOptional

Methods

Name Description
toString Returns a string representation of this construct.
toString 
public toString(): string

Returns a string representation of this construct.

Static Functions

Name Description
isConstruct Checks if x is a construct.
isConstruct 
import { SitemapsConstruct } from '@shutterstock/sitemaps-cdk'

SitemapsConstruct.isConstruct(x: any)

Checks if x is a construct.

Use this method instead of instanceof to properly detect Construct instances, even when the construct library is symlinked.

Explanation: in JavaScript, multiple copies of the constructs library on disk are seen as independent, completely different libraries. As a consequence, the class Construct in each copy of the constructs library is seen as a different class, and an instance of one class will not test as instanceof the other class. npm install will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the constructs library can be accidentally installed, and instanceof will behave unpredictably. It is safest to avoid using instanceof, and using this type-testing method instead.

xRequired
  • Type: any

Any object.

Properties

Name Type Description
node constructs.Node The tree node.
indexWriterLambdaFunction aws-cdk-lib.aws_lambda.Function Index Writer Lambda Function.
kinesisInputStream aws-cdk-lib.aws_kinesis.IStream Kinesis Input Stream for Sitemap Items.
s3SitemapsBucket aws-cdk-lib.aws_s3.IBucket No description.
sitemapWriterLambdaFunction aws-cdk-lib.aws_lambda.Function Sitemap Writer Lambda Function.
dynamoDBTable aws-cdk-lib.aws_dynamodb.Table No description.
nodeRequired 
public readonly node: Node;
  • Type: constructs.Node

The tree node.

indexWriterLambdaFunctionRequired 
public readonly indexWriterLambdaFunction: Function;
  • Type: aws-cdk-lib.aws_lambda.Function

Index Writer Lambda Function.

kinesisInputStreamRequired 
public readonly kinesisInputStream: IStream;
  • Type: aws-cdk-lib.aws_kinesis.IStream

Kinesis Input Stream for Sitemap Items.

s3SitemapsBucketRequired 
public readonly s3SitemapsBucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket
sitemapWriterLambdaFunctionRequired 
public readonly sitemapWriterLambdaFunction: Function;
  • Type: aws-cdk-lib.aws_lambda.Function

Sitemap Writer Lambda Function.

dynamoDBTableOptional 
public readonly dynamoDBTable: Table;
  • Type: aws-cdk-lib.aws_dynamodb.Table

Structs

SitemapFreshnerConstructProps

Initializer 

import { SitemapFreshnerConstructProps } from '@shutterstock/sitemaps-cdk'

const sitemapFreshnerConstructProps: SitemapFreshnerConstructProps = { ... }

Properties

Name Type Description
dynamodbTable aws-cdk-lib.aws_dynamodb.ITable Existing DynamoDB Table to be used for freshening sitemap files.
s3SitemapsBucket aws-cdk-lib.aws_s3.IBucket Existing S3 sitemap files bucket as the desintation for the freshner Lambda.
autoDeleteEverything boolean Automatically clean up durable resources (e.g. for PR builds).
env string Environment used to select config files from the config layer, if provided.
kinesisEventSourceExtraProps aws-cdk-lib.aws_lambda_event_sources.KinesisEventSourceProps Extra Kinesis event source properties (e.g. batch size).
kinesisInputStream aws-cdk-lib.aws_kinesis.IStream Kinesis input stream.
lambdaFuncFreshenerExtraProps aws-cdk-lib.aws_lambda.FunctionOptions Extra properties for the sitemap freshener Lambda function.
lambdaFuncFreshenerRuntime aws-cdk-lib.aws_lambda.Runtime Optional runtime for the sitemap freshener Lambda function.
metricsNamespace string Metrics namespace for Sitemap Freshener Lambda.
s3SitemapsPrefix string Path to write sitemap files to.
testBuild boolean Disable minification during test builds so that snaphots match.
dynamodbTableRequired 
public readonly dynamodbTable: ITable;
  • Type: aws-cdk-lib.aws_dynamodb.ITable

Existing DynamoDB Table to be used for freshening sitemap files.

s3SitemapsBucketRequired 
public readonly s3SitemapsBucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket

Existing S3 sitemap files bucket as the desintation for the freshner Lambda.

autoDeleteEverythingOptional 
public readonly autoDeleteEverything: boolean;
  • Type: boolean
  • Default: false

Automatically clean up durable resources (e.g. for PR builds).

⚠️ CAUTION: This will delete the S3 bucket (and all contents) and DynamoDB table.

envOptional 
public readonly env: string;
  • Type: string
  • Default: none

Environment used to select config files from the config layer, if provided.

Passed to the Lamba as NODE_CONFIG_ENV

Example

'prod'
kinesisEventSourceExtraPropsOptional 
public readonly kinesisEventSourceExtraProps: KinesisEventSourceProps;
  • Type: aws-cdk-lib.aws_lambda_event_sources.KinesisEventSourceProps

Extra Kinesis event source properties (e.g. batch size).

kinesisInputStreamOptional 
public readonly kinesisInputStream: IStream;
  • Type: aws-cdk-lib.aws_kinesis.IStream

Kinesis input stream.

If not provided no event source will be created; the lambda can be invoked manually with a start operation using the Kinesis message format.

lambdaFuncFreshenerExtraPropsOptional 
public readonly lambdaFuncFreshenerExtraProps: FunctionOptions;
  • Type: aws-cdk-lib.aws_lambda.FunctionOptions
  • Default: { architecture: ARM_64, memorySize: 2000, timeout: Duration.minutes(15), logRetention: logs.RetentionDays.ONE_MONTH }

Extra properties for the sitemap freshener Lambda function.

Configuration can be passed either via Environment Variables or via a config file layer that can have a default config.yml and an NODE_CONFIG_ENV-specific config override file: config-${NODE_CONFIG_ENV}.yml.

Example

{ functionName: 'my-function-name', layers: lambda.Code.fromAsset('./configs/sitemap-freshner/') }
lambdaFuncFreshenerRuntimeOptional 
public readonly lambdaFuncFreshenerRuntime: Runtime;
  • Type: aws-cdk-lib.aws_lambda.Runtime
  • Default: lambda.Runtime.NODEJS_20_X

Optional runtime for the sitemap freshener Lambda function.

metricsNamespaceOptional 
public readonly metricsNamespace: string;
  • Type: string

Metrics namespace for Sitemap Freshener Lambda.

s3SitemapsPrefixOptional 
public readonly s3SitemapsPrefix: string;
  • Type: string
  • Default: defers to config file or sitemaps/ if not in config file

Path to write sitemap files to.

This should be unique per durable deployment, such as:

  • sitemaps/20210704/
  • sitemaps/20211031/

When a new durable deployment is created, by changing the stack name, this prefix should be changed so that the new backfill writes to a new path and the old stack, with the old Lambda code, continues to write to the old path. When it is desired to migrate to the new sitemaps, a 301 should be returned for the well-known sitemap index URL to point to the deploy-specific sitemap index URL.

testBuildOptional 
public readonly testBuild: boolean;
  • Type: boolean
  • Default: false

Disable minification during test builds so that snaphots match.

SitemapsConstructProps

Initializer 

import { SitemapsConstructProps } from '@shutterstock/sitemaps-cdk'

const sitemapsConstructProps: SitemapsConstructProps = { ... }

Properties

Name Type Description
autoDeleteEverything boolean Automatically clean up durable resources (e.g. for PR builds).
dynamodbTable aws-cdk-lib.aws_dynamodb.ITable Optional - Already-existing DynamoDB Table to be used for sitemap and index files.
dynamodbTableName string Optional - Name of the DynamoDB table.
dynamodbTableProps aws-cdk-lib.aws_dynamodb.TableProps Optional - Extra properties for the DynamoDB table.
env string Environment used to select config files from the config layer, if provided.
kinesisInputStreamName string Specific name of the kinesis input stream.
kinesisInputStreamShards number Number of shards in the kinesis input stream.
kinesisSitemapWriterRetentionDays number Number of days of data retention in the kinesis sitemap writer stream.
lambdaFuncIndexWriterExtraProps aws-cdk-lib.aws_lambda.FunctionOptions Extra properties for the sitemap writer Lambda function.
lambdaFuncIndexWriterRuntime aws-cdk-lib.aws_lambda.Runtime Optional runtime for the index writer Lambda function.
lambdaFuncSitemapWriterExtraProps aws-cdk-lib.aws_lambda.FunctionOptions Extra properties for the sitemap writer Lambda function.
lambdaFuncSitemapWriterRuntime aws-cdk-lib.aws_lambda.Runtime Optional runtime for the sitemap writer Lambda function.
metricsIndexWriterName string Metrics namespace and name for Index Writer Lambda.
metricsSitemapWriterName string Metrics namespace and name for Sitemap Writer Lambda.
s3SitemapsBucket aws-cdk-lib.aws_s3.IBucket Optional - Already-existing S3 sitemaps files bucket to be used for sitemap and index files.
s3SitemapsBucketName string Optional - Specific name of the S3 sitemaps files bucket to be created.
s3SitemapsPrefix string Path to write sitemap files to.
sitemapWriterBatchSize number Batch Size for Sitemap Writer.
sitemapWriterMaxBatchingWindow aws-cdk-lib.Duration Max Batching Window Duration for Sitemap Writer.
testBuild boolean Disable minification during test builds so that snaphots match.
autoDeleteEverythingOptional 
public readonly autoDeleteEverything: boolean;
  • Type: boolean
  • Default: false

Automatically clean up durable resources (e.g. for PR builds).

⚠️ CAUTION: This will delete the S3 bucket (and all contents) and DynamoDB table.

dynamodbTableOptional 
public readonly dynamodbTable: ITable;
  • Type: aws-cdk-lib.aws_dynamodb.ITable
  • Default: DynamoDB Table will be created

Optional - Already-existing DynamoDB Table to be used for sitemap and index files.

If not set, a table will be created.

dynamodbTableNameOptional 
public readonly dynamodbTableName: string;
  • Type: string
  • Default: auto assigned by CDK

Optional - Name of the DynamoDB table.

dynamodbTablePropsOptional 
public readonly dynamodbTableProps: TableProps;
  • Type: aws-cdk-lib.aws_dynamodb.TableProps

Optional - Extra properties for the DynamoDB table.

envOptional 
public readonly env: string;
  • Type: string
  • Default: none

Environment used to select config files from the config layer, if provided.

Passed to the Lamba as NODE_CONFIG_ENV

Example

'prod'
kinesisInputStreamNameOptional 
public readonly kinesisInputStreamName: string;
  • Type: string
  • Default: auto assigned

Specific name of the kinesis input stream.

If not specified, the name will be availabe via the kinesisInputStreamName export

kinesisInputStreamShardsOptional 
public readonly kinesisInputStreamShards: number;
  • Type: number
  • Default: 4

Number of shards in the kinesis input stream.

kinesisSitemapWriterRetentionDaysOptional 
public readonly kinesisSitemapWriterRetentionDays: number;
  • Type: number
  • Default: 1 day

Number of days of data retention in the kinesis sitemap writer stream.

lambdaFuncIndexWriterExtraPropsOptional 
public readonly lambdaFuncIndexWriterExtraProps: FunctionOptions;
  • Type: aws-cdk-lib.aws_lambda.FunctionOptions
  • Default: { architecture: ARM_64, memorySize: 2000, timeout: Duration.minutes(15), logRetention: logs.RetentionDays.ONE_MONTH }

Extra properties for the sitemap writer Lambda function.

Configuration can be passed either via Environment Variables or via a config file layer that can have a default config.yml and an NODE_CONFIG_ENV-specific config override file: config-${NODE_CONFIG_ENV}.yml.

Example

{ functionName: 'my-index-function-name', layers: lambda.Code.fromAsset('./configs/index-writer/') }
lambdaFuncIndexWriterRuntimeOptional 
public readonly lambdaFuncIndexWriterRuntime: Runtime;
  • Type: aws-cdk-lib.aws_lambda.Runtime
  • Default: lambda.Runtime.NODEJS_20_X

Optional runtime for the index writer Lambda function.

lambdaFuncSitemapWriterExtraPropsOptional 
public readonly lambdaFuncSitemapWriterExtraProps: FunctionOptions;
  • Type: aws-cdk-lib.aws_lambda.FunctionOptions
  • Default: { architecture: ARM_64, memorySize: 2000, timeout: Duration.minutes(15), logRetention: logs.RetentionDays.ONE_MONTH }

Extra properties for the sitemap writer Lambda function.

Configuration can be passed either via Environment Variables or via a config file layer that can have a default config.yml and an NODE_CONFIG_ENV-specific config override file: config-${NODE_CONFIG_ENV}.yml.

Example

{ functionName: 'my-sitemap-function-name', layers: lambda.Code.fromAsset('./configs/sitemap-writer/') }
lambdaFuncSitemapWriterRuntimeOptional 
public readonly lambdaFuncSitemapWriterRuntime: Runtime;
  • Type: aws-cdk-lib.aws_lambda.Runtime
  • Default: lambda.Runtime.NODEJS_20_X

Optional runtime for the sitemap writer Lambda function.

metricsIndexWriterNameOptional 
public readonly metricsIndexWriterName: string;
  • Type: string
  • Default: none, metrics disabled

Metrics namespace and name for Index Writer Lambda.

metricsSitemapWriterNameOptional 
public readonly metricsSitemapWriterName: string;
  • Type: string
  • Default: none, metrics disabled

Metrics namespace and name for Sitemap Writer Lambda.

s3SitemapsBucketOptional 
public readonly s3SitemapsBucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket
  • Default: S3 Bucket will be created

Optional - Already-existing S3 sitemaps files bucket to be used for sitemap and index files.

If not set, a bucket will be created.

s3SitemapsBucketNameOptional 
public readonly s3SitemapsBucketName: string;
  • Type: string
  • Default: auto assigned

Optional - Specific name of the S3 sitemaps files bucket to be created.

If not set, a name will be assigned.

s3SitemapsPrefixOptional 
public readonly s3SitemapsPrefix: string;
  • Type: string
  • Default: defers to config file or sitemaps/ if not in config file

Path to write sitemap files to.

This should be unique per durable deployment, such as:

  • sitemaps/20210704/
  • sitemaps/20211031/

When a new durable deployment is created, by changing the stack name, this prefix should be changed so that the new backfill writes to a new path and the old stack, with the old Lambda code, continues to write to the old path. When it is desired to migrate to the new sitemaps, a 301 should be returned for the well-known sitemap index URL to point to the deploy-specific sitemap index URL.

sitemapWriterBatchSizeOptional 
public readonly sitemapWriterBatchSize: number;
  • Type: number
  • Default: 10,000

Batch Size for Sitemap Writer.

This is the maximum number of records that will be processed in a single batch.

Keep this number high (e.g. 10,000) to reduce the number of Lambda invocation under high load and to reduce the number of times that the latest XML file is read from S3, deserialized, updated, serialized, and written back to S3, which is time consuming (10-30 seconds) and gets longer as the file approaches being full.

sitemapWriterMaxBatchingWindowOptional 
public readonly sitemapWriterMaxBatchingWindow: Duration;
  • Type: aws-cdk-lib.Duration
  • Default: 30 seconds

Max Batching Window Duration for Sitemap Writer.

Allowed range is 0-300 seconds. It is not suggested to set this to 0 seconds as that will parse the XML file far too often and will likely result in the Lambda function falling behind on reading the Kinesis Stream.

testBuildOptional 
public readonly testBuild: boolean;
  • Type: boolean
  • Default: false

Disable minification during test builds so that snaphots match.

Protocols

ISitemapsConstruct

Properties

Name Type Description
indexWriterLambdaFunction aws-cdk-lib.aws_lambda.Function Index Writer Lambda Function.
kinesisInputStream aws-cdk-lib.aws_kinesis.IStream Kinesis Input Stream for Sitemap Items.
s3SitemapsBucket aws-cdk-lib.aws_s3.IBucket No description.
sitemapWriterLambdaFunction aws-cdk-lib.aws_lambda.Function Sitemap Writer Lambda Function.
dynamoDBTable aws-cdk-lib.aws_dynamodb.Table No description.
indexWriterLambdaFunctionRequired 
public readonly indexWriterLambdaFunction: Function;
  • Type: aws-cdk-lib.aws_lambda.Function

Index Writer Lambda Function.

kinesisInputStreamRequired 
public readonly kinesisInputStream: IStream;
  • Type: aws-cdk-lib.aws_kinesis.IStream

Kinesis Input Stream for Sitemap Items.

s3SitemapsBucketRequired 
public readonly s3SitemapsBucket: IBucket;
  • Type: aws-cdk-lib.aws_s3.IBucket
sitemapWriterLambdaFunctionRequired 
public readonly sitemapWriterLambdaFunction: Function;
  • Type: aws-cdk-lib.aws_lambda.Function

Sitemap Writer Lambda Function.

dynamoDBTableOptional 
public readonly dynamoDBTable: Table;
  • Type: aws-cdk-lib.aws_dynamodb.Table