Skip to content

Terraform configuration for creating RabbitMQ cluster on AWS.

License

Notifications You must be signed in to change notification settings

smartrent/terraform-aws-rabbitmq

 
 

Repository files navigation

Dead simple Terraform configuration for creating RabbitMQ cluster on AWS

What it does ?

  1. Uses official RabbitMQ docker image.
  2. Creates N nodes in M subnets
  3. Creates Autoscaling Group and ELB to load balance nodes
  4. Makes sure nodes can talk to each other and create cluster
  5. Make sure new nodes attempt to join the cluster at startup
  6. Configures / vhost queues in High Available (Mirrored) mode with automatic synchronization ("ha-mode":"all", "ha-sync-mode":"3")
  7. Installs and configures Datadog Agent to gather metrics and logs for RabbitMQ

How to use it ?

Copy and paste into your Terraform configuration:

module "rabbitmq" {
  source                            = "github.com/smartrent/terraform-aws-rabbitmq"
  vpc_id                            = var.vpc_id
  ssh_key_name                      = var.ssh_key_name
  subnet_ids                        = var.subnet_ids
  elb_additional_security_group_ids = [var.cluster_security_group_id]
  min_size                          = "3"
  max_size                          = "3"
  desired_size                      = "3"
  dd_env                            = var.env_name
  dd_site                           = var.datadog_site
  kms_key_arn                       = var.kms_key_id
  ecr_registry_id                   = var.ecr_registry_id
  rabbitmq_image                    = var.rabbitmq_image
  ami_id                            = var.ami_id
}

Need an AMI ID? Use empty string ("") to allow the module to get the latest for you. Remember to replace the AMI_ID with a static value as latest changes frequently and will ultimately cause perpetual drift.

Then run terraform init, terraform plan and terraform apply.

Are 3 node not enough ? Update sizes to 5 and run terraform apply again, it will update Autoscaling Group and add 2 nodes more. Dead simple.

Node becomes unresponsive ? Autoscaling group and ELB Health Checks will automatically replace it with new one, without data loss.

Note: The VPC must have enableDnsHostnames = true and enableDnsSupport = true for the private DNS names to be resolvable for the nodes to connect to each other.

Debugging

If you can SSH onto one of the nodes you can run: docker exec rabbitmq rabbitmqctl cluster_status to see the cluster status of that node.

Upgrading

Sometimes we will need to do a hot restart of a node in the cluster in order to preform some maintenance, upgrade, or infrastructure improvements. To do this graceful we need to remove the current node from the cluster. This helps keep the node in sync and organized correctly. To do this we need to stop the app and reset the node as follows:

docker exec rabbitmq rabbitmqctl cluster_status
docker exec rabbitmq rabbitmqctl stop_app
docker exec rabbitmq rabbitmqctl reset
docker exec rabbitmq rabbitmqctl cluster_status

README.md updated successfully

Requirements

Name Version
terraform >= 1
aws >= 4.0

Providers

Name Version
aws >= 4.0
cloudinit n/a
random n/a

Modules

No modules.

Resources

Name Type
aws_autoscaling_group.rabbitmq resource
aws_elb.elb resource
aws_iam_instance_profile.iam_profile resource
aws_iam_policy.ssm_managed_instances resource
aws_iam_role.iam_role resource
aws_iam_role_policy.iam_policy resource
aws_iam_role_policy_attachment.inspector2_managed_cis_policy resource
aws_iam_role_policy_attachment.ssm_managed_instance_core resource
aws_launch_template.rabbitmq resource
aws_security_group.rabbitmq_elb resource
aws_security_group.rabbitmq_nodes resource
aws_ssm_parameter.datadog_api_key resource
aws_ssm_parameter.datadog_user_password resource
aws_ssm_parameter.rabbit_admin_password resource
aws_ssm_parameter.rabbit_password resource
aws_ssm_parameter.secondary_datadog_user_password resource
aws_ssm_parameter.secondary_rabbit_admin_password resource
aws_ssm_parameter.secondary_rabbit_password resource
aws_ssm_parameter.secret_cookie resource
random_password.admin_password resource
random_password.datadog_password resource
random_password.rabbit_password resource
random_password.secondary_admin_password resource
random_password.secondary_datadog_password resource
random_password.secondary_rabbit_password resource
random_password.secret_cookie resource
aws_ami.amazon_linux_2 data source
aws_ami.amazon_linux_2_latest data source
aws_iam_policy_document.policy_doc data source
aws_iam_policy_document.policy_permissions_doc data source
aws_iam_policy_document.ssm_managed_instances data source
aws_region.current data source
cloudinit_config.user_data data source

Inputs

Name Description Type Default Required
access_log_bucket optional bucket name to use for access logs string "bucketname" no
access_log_bucket_prefix optional prefix to use for access logs string "" no
access_log_interval How often for the ELB to publish access logs in minutes number 60 no
access_logs_enabled Whether or not to enable access logging on the ELB bool false no
ami_id The AMI ID to use for the ec2 instance. Empty string uses a pre-configured latest. string "" no
aws_session_manager_enabled Whether or not the ec2 instances in this cluster should allow session manager permissions bool false no
dd_env The environment the app is running in string n/a yes
dd_site The Datadog site url string n/a yes
desired_size Desired number of RabbitMQ nodes number 2 no
ecr_registry_id The ECR registry ID string n/a yes
elb_additional_security_group_ids List of additional ELB security group ids list(string) [] no
enable_password_rotation Whether to configure secondary users to facilitate password rotations. bool false no
encrypted_ebs_instance_volume Whether to encrypt the instance ebs volume bool true no
health_check_grace_period The ASG health check grace period number 400 no
health_check_interval The ELB health check interval in seconds number 30 no
healthy_threshold The ELB health check healthy threshold count number 2 no
inspector2_cis_scanning_enabled Whether to attach required policies for inspector 2 ec2 CIS scans bool true no
instance_type The EC2 instance type to use string "m5.large" no
instance_volume_iops The amount of provisioned iops number 0 no
instance_volume_size The size of the instance volume in gigabytes number 0 no
instance_volume_type The instance volume type to use (standard, gp2, gp3, st1, sc1, io1) string "standard" no
kms_key_arn The KMS key arn to use for encrypting and decrypting SSM parameters string n/a yes
max_size Maximum number of RabbitMQ nodes number 2 no
min_size Minimum number of RabbitMQ nodes number 2 no
name The name of the RabbitMQ cluster string "main" no
nodes_additional_security_group_ids List of additional node security group ids list(string) [] no
rabbitmq_image The RabbitMQ docker image string n/a yes
session_manager_kms_encryption_enabled Whether to enable session manager kms encryption to protect session data bool false no
session_manager_kms_key_arn The kms key arn to use for session manager session encryption string "" no
ssh_key_name The ssh key to provide the instance to use for ssh login string n/a yes
subnet_ids Subnets for RabbitMQ nodes list(string) n/a yes
tags Optional additional Tags to add onto resources this module creates map(string) {} no
timeout The ELB health check length of time before timeout in seconds number 3 no
unhealthy_threshold The ELB health check unhealthy threshold count number 10 no
upgrading_minor_version_number The value of the minor version you are upgrading to. number 0 no
use_secondary_users Whether the instance should use the secondary users. bool false no
vpc_id n/a any n/a yes

Outputs

Name Description
admin_password n/a
amqp_url n/a
iam_profile_arn n/a
iam_role n/a
rabbit_password n/a
rabbitmq_elb_dns n/a
secondary_admin_password n/a
secondary_amqp_url n/a
secondary_rabbit_password n/a
secret_cookie n/a
ssm_managed_instances_policy_arn n/a

About

Terraform configuration for creating RabbitMQ cluster on AWS.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • HCL 100.0%