Overview

Terraform deployment scripts for Beekeeper.

Includes support for deploying Beekeeper on ECS and Kubernetes. Also includes deployment scripts for a Lambda which notifies Slack when Beekeeper's dead letter queue receives a message (this module is not required to run Beekeeper).

Dependencies

A database password is defined in aws-secrets-manager. The terraform module will use that password for the database (see property k8s_db_password_secret ).

Providers

Name	Version
archive	n/a
aws	n/a
kubernetes	n/a
random	n/a
template	n/a

Inputs

Name	Description	Type	Default	Required
allowed_s3_buckets	List of S3 Buckets to which Beekeeper will have read-write access.	list(string)	[]	no
apiary_metastore_listener_arn	ARN of the Apiary Metastore Listener.	string	n/a	yes
aws_region	AWS region to use for resources.	string	n/a	yes
beekeeper_tags	A map of tags to apply to resources.	map(string)	n/a	yes
path_cleanup_docker_image	Beekeeper Path Cleanup docker image.	string	"expediagroup/beekeeper-path-cleanup"	no
path_cleanup_docker_image_version	Beekeeper Path Cleanup docker image version.	string	"latest"	no
path_cleanup_ecs_cpu	The amount of CPU used to allocate for the Beekeeper Path Cleanup ECS task. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html	number	2048	no
path_cleanup_ecs_memory	The amount of memory (in MiB) used to allocate for the Beekeeper Path Cleanup container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html	number	4096	no
metadata_cleanup_docker_image	Beekeeper Metadata Cleanup docker image.	string	"expediagroup/beekeeper-metadata-cleanup"	no
metadata_cleanup_docker_image_version	Beekeeper Metadata Cleanup docker image version.	string	"latest"	no
db_backup_retention	The number of days to retain backups for the RDS Beekeeper DB.	number	10	no
db_backup_window	Preferred backup window for the RDS Beekeeper DB in UTC.	string	"02:00-03:00"	no
db_maintenance_window	Preferred maintenance window for the RDS Beekeeper DB in UTC.	string	"wed:03:00-wed:04:00"	no
db_apply_immediately	Specifies whether any database modifications are applied immediately, or during the next maintenance window.	bool	false	no
db_performance_insights_enabled	Specifies whether Performance Insights are enabled.	bool	true	no
db_password_key	Key to acquire the database password for the strategy specified.	string	n/a	yes
db_username	Username for the master DB user.	string	"beekeeper"	no
docker_registry_auth_secret_name	Docker Registry authentication SecretManager secret name.	string	""	no
path_cleanup_dry_run_enabled	Enable Path Cleanup to perform dry runs of deletions only.	string	"false"	no
metadata_cleanup_dry_run_enabled	Enable Metadata Cleanup to perform dry runs of deletions only.	string	"false"	no
graphite_enabled	Enable to produce Graphite metrics - true or false.	string	"false"	no
graphite_host	Graphite metrics host.	string	"localhost"	no
graphite_port	Graphite port.	number	2003	no
graphite_prefix	Prefix for Graphite metrics.	string	""	no
metastore_uri	URI of the metastore where tables to be cleaned-up are located. Required for Beekeeper Metadata Cleanup.	string	""	yes
instance_name	Beekeeper instance name to identify resources in multi-instance deployments.	string	""	no
instance_type	Service to run Beekeeper on. Supported services: ecs (default), k8s. Leaving this blank will still deploy auxiliary components (e.g. RDS, SQS etc.).	string	"ecs"	no
k8s_app_name	Name to give to all Kubernetes resources that are deployed.	string	"beekeeper"	no
k8s_path_cleanup_cpu	Total cpu to allot to the Beekeeper Path Cleanup pod.	string	"500m"	no
k8s_path_cleanup_ingress_host	Ingress host name for Beekeeper Path Cleanup.	string	""	no
k8s_path_cleanup_ingress_path	Ingress path regex for Beekeeper Path Cleanup.	string	""	no
k8s_path_cleanup_liveness_delay	Liveness delay (in seconds) for the Beekeeper Path Cleanup service.	number	60	no
k8s_path_cleanup_memory	Total memory to allot to the Beekeeper Path Cleanup pod.	string	"2Gi"	no
k8s_path_cleanup_port	Internal port that the Beekeeper Path Cleanup service runs on.	number	8008	no
k8s_metadata_cleanup_cpu	Total cpu to allot to the Beekeeper Metadata Cleanup pod.	string	"500m"	no
k8s_metadata_cleanup_ingress_host	Ingress host name for Beekeeper Metadata Cleanup.	string	""	no
k8s_metadata_cleanup_ingress_path	Ingress path regex for Beekeeper Metadata Cleanup.	string	""	no
k8s_metadata_cleanup_liveness_delay	Liveness delay (in seconds) for the Beekeeper Metadata Cleanup service.	number	60	no
k8s_metadata_cleanup_memory	Total memory to allot to the Beekeeper Metadata Cleanup pod.	string	"2Gi"	no
k8s_metadata_cleanup_port	Internal port that the Beekeeper Metadata Cleanup service runs on.	number	9008	no
k8s_image_pull_policy	Policy for the Kubernetes orchestrator to pull images.	string	"Always"	no
k8s_ingress_enabled	Boolean flag to determine if we should create an ingress or not. (0 = off, 1 = on).	number	0	no
k8s_ingress_tls_hosts	List of hosts for TLS configuration of a Kubernetes ingress.	list(string)	[]	no
k8s_ingress_tls_secret	Secret name for TLS configuration of a Kubernetes ingress.	string	""	no
k8s_kiam_role_arn	KIAM role arn to use for creating a K8S IAM role with the correct assume role permissions.	string	""	no
k8s_db_password_secret	Name of the Kubernetes secret that would store the db password for beekeeper.	string	"beekeeper-db-password"	no
k8s_namespace	Namespace to deploy all Kubernetes resources to.	string	"beekeeper"	no
k8s_node_affinity	Full node_affinity object as per terraform/Kubernetes docs.	object({})	{}	no
k8s_node_selector	Full node_selector object as per terraform/Kubernetes docs.	object({})	{}	no
k8s_node_tolerations	Full k8s_node_tolerations object as per terraform/Kubernetes docs.	object({})	{}	no
k8s_scheduler_apiary_cpu	Total cpu to allot to the Beekeeper Scheduler Apiary pod.	string	"500m"	no
k8s_scheduler_apiary_ingress_host	Ingress host name for Beekeeper Scheduler Apiary.	string	""	no
k8s_scheduler_apiary_ingress_path	Ingress path regex for Beekeeper Scheduler Apiary.	string	""	no
k8s_scheduler_apiary_liveness_delay	Liveness delay (in seconds) for the Beekeeper Scheduling service.	number	60	no
k8s_scheduler_apiary_memory	Total memory to allot to the Beekeeper Scheduler Apiary pod.	string	"2Gi"	no
k8s_scheduler_apiary_port	Internal port that the Beekeeper Scheduler Apiary service runs on.	number	8080	no
message_retention_seconds	SQS message retention (s).	number	604800	no
scheduler_apiary_docker_image	Beekeeper Scheduler Apiary image.	string	"expediagroup/beekeeper-scheduler-apiary"	no
scheduler_apiary_docker_image_version	Beekeeper Scheduler Apiary image version.	string	"latest"	no
scheduler_apiary_ecs_cpu	The amount of CPU used to allocate for the Beekeeper Scheduler Apiary ECS task. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html	number	2048	no
scheduler_apiary_ecs_memory	The amount of memory (in MiB) used to allocate for the Beekeeper Scheduler Apiary container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html	number	4096	no
k8s_api_memory	Total memory to allot to the Beekeeper API pod	string	"1Gi"	no
k8s_api_cpu	Total cpu to allot to the Beekeeper API pod	string	"500m"	no
k8s_beekeeper_api_port	Internal port that the Beekeeper API service runs on	number	7008	yes
k8s_api_liveness_delay	Liveness delay (in seconds) for the Beekeeper API service	number	60	no
k8s_api_ingress_host	Ingress host name for Beekeeper API	string	""	no
k8s_api_ingress_path	Ingress path regex for Beekeeper API	string	""	no
api_docker_image	Beekeeper API image	string	"expediagroup/beekeeper-api"	yes
api_docker_image_version	Beekeeper API docker image version	string	"latest"	no
prometheus_enabled	Enable to pull metrics using Prometheus - true or false.	string	"false"	no
queue_name	Beekeeper SQS Queue name.	string	"apiary-beekeeper"	no
queue_stale_messages_timeout	Beekeeper SQS Queue Cloudwatch Alert timeout for messages older than this number of seconds.	number	1209600	no
rds_allocated_storage	RDS allocated storage in GBs.	string	10	no
rds_engine_version	RDS engine version.	string	"8.0"	no
rds_instance_class	RDS instance class.	string	"db.t2.micro"	no
rds_max_allocated_storage	RDS max allocated storage (autoscaling) in GBs.	string	100	no
rds_parameter_group_name	RDS parameter group.	string	"default.mysql8.0"	no
rds_storage_type	RDS storage type.	string	"gp3"	no
rds_subnets	Subnets in which to provision Beekeeper RDS DB.	list(string)	n/a	yes
receive_wait_time_seconds	SQS receive wait time (s).	number	20	no
scheduler_apiary_delay_ms	Delay between each cleanup job that is scheduled in milliseconds.	number	300000	no
slack_channel	Slack channel to which alerts about messages landing on the dead letter queue should be sent.	string	""	no
slack_lambda_enabled	Boolean flag to determine if Beekeeper should create a Slack notifying Lambda for the dead letter queue. (0 = off, 1 = on).	number	0	no
slack_webhook_url	Slack URL to which alerts about messages landing on the dead letter queue should be sent.	string	""	no
subnets	Subnets in which to install Beekeeper.	list(string)	n/a	yes
vpc_id	VPC in which to install Beekeeper.	string	n/a	yes

Outputs

No output.

Legal

This project is available under the Apache 2.0 License.

Copyright 2019 Expedia, Inc.