Kentik AWS onboarding TF module to operate on multiple AWS regions

[danielfurman]

Context

Kentik customer attempted to use our Terraform integration to create cloudexports from deployments in multiple AWS regions. They have resources in one AWS account deployed in multiple region. This is a different scenario than in KNTK-155. The problem with multi-region scenario is that AWS IAM has global scope, so roles and (I assume also) policies need to be created in only one region, while Kentik cloudexports has to be created separately for each region. The customer was able to work around the problem by creating a wrapper which creates roles and policies with different names in each region, but that complicates maintenance and auditing.

Goal

Allow creation of AWS resources and Kentik cloudexports for customer single AWS account with deployments in several regions in one execution of terraform apply.

Proposed solution

Extend the region variable to be a list allowing to specify the AWS regions to which the config should be applied. The AWS IAM resources should be created only in the first region on the list. S3 buckets likely need to be created separately for each region using a unique prefix in the name, because S3 has global namespace.

This is really just a proposal and exact details of the solution need further research and testing.

Additional context (from Jira comments)

Comment 1

Identity of Kentik flow exporter (internal name Transfluo) is the same in all AWS regions. This means that IAM roles and policies required for allowing Transfluo to access S3 buckets and AWS API (to collect metada) shall be configured only once and shared by deployments in all regions.

The procedure should allow future extensions for additional 2 use cases:

1. flow logging to S3 buckets owned by Kentik
2. replication of flow log data from customer owner S3 buckets to S3 buckets owned by Kentik.

Comment 2

The process needs to:

1. for each target region create S3 buckets and other AWS region specific resources and also cloudexport in Kentik

2. create roles and policies that apply to all regions and VPCs from which cloud exports are to be configured. That shall be done once, ideally in a region that the customer can choose (I don’t know where the region matters, but it is good to have control over this aspect, just in case it did)

I’m not entirely sure what you mean by “module” in this context. All of the machinery needs to be re-usable for any number of regions and VPCs. List of target regions shall be provided as input parameter. It probably needs to be outside of a TF resource file, because of the need to use different configuration of AWS provider for each target region. If this is really the case, the mechanics should be the same (or as similar as possible) as for Azure multi-subscription onboarding.
I think that the per region actions can be one (sub-)module and global actions another one (I’m though not a TF expert). We likely will need an external wrapper which iterates over regions, creates region specific resource files and switches user identity to the one appropriate for the target region. Finally in the last step it calls the “global” (sub-)module which creates IAM configuration (roles and policies). Again, if this is needed, the mechanics should be as similar to Azure as possible.

Solution in progress

Open PR: #56