Skip to content

ECR repos, lifecycle policies, and their associated OIDC authentication for github repos publishing to them

License

Notifications You must be signed in to change notification settings

MITLibraries/mitlib-tf-workloads-ecr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is a public repo

Centralized ECR Repository Creation

This repo builds the ECR (Elastic Container Registry) repositories for containers. Most of the heavy lifting is in an embedded module modules/ecr/main.tf.

The login policy can be shared between each app, since it's the same no matter what.

Dependencies

The only dependency is the ARN of the OpenID Connect Provider (placed in Parameter Store by the mitlib-tf-workloads-init repo).

Pre-Commit Hooks

For proper linting and checking, this repo uses pre-commit hooks. The following should be installed in the local workstation

After the first checkout locally, run the following command to initialize the pre-commit hooks.

pre-commit install  --hook-type pre-push

It is possible to run the pre-commit hooks manually. To run all the pre-commit hooks for this repo, run

pre-commit run --all-files

To run just the checkov checker, run

pre-commit run checkov

To run just the terraform-docs hook to update the README, run

pre-commit run terraform-docs-go

See .pre-commit-config.yaml for other pre-commit hooks that can be run.

Usage

There is a tight relationship between ECR repositories created here and the associated application repositories in GitHub due to the use of OIDC in the GitHub Actions in those application repositories. Make sure to coordinate any new ECR repositories with the developers building the applications that will be published there.

The "app-repo" tag should correspond with the name of the repo that contains the code that is compiled and saved in the ECR, or a combination of infrastructure and app repo names if multiple ECR's are needed in the same infrastructure project.

For each new ECR repository (or linked collection of ECR repositories), create a new ..._ecr.tf file that contains

  • at least one module call to the embedded ecr module to actually create the ECR
  • at least one set of four outputs (four outputs per ECR)
    • a Makefile
    • a dev_build caller workflow
    • a stage_build caller workflow
    • a prod_promote caller workflow

The ppod_ecr.tf is a good example of a single ECR repository for a Lambda function build around a containerized app. The timdex_ecrs.tf is a good example of a collection of ECR repositories all linked to one project (and there are both Fargate-linked ECRs and Lambda-linked ECRs in that file).

Note: For Lambda function ECRs, it is imperative that the Infra engineer coordinates with the software engineer to determine the name of the Lambda function as part of the creation of the ECR by this repository.

For Application Developers

A quick note for application developers and the integration of workflows to automate the deployment of their containerized application to either Fargate or Lambda. When this code is deployed in Terraform Cloud, it generates outputs that contain the caller workflows code as well as the Makefile code for their application. Those outputs are accessible to the developers via Terraform Cloud -- they can go into TfC, find the correct Terraform Output, and then copy that text into their application repository.

Making this work in your environment outside of MIT libraries

This repository is a part of an ecosystem of components designed to work in our AWS organization. This component is responsible for a standardized setup of ECR repositories and a build process that goes in Github Actions and Makefiles. On its own, this repository could be useful to you if you want to emulate how we deploy and promote containers across our AWS accounts, or utilize GitHub OIDC connections for depositing ECR containers to AWS. Before this will deploy in your environment, you will need an OpenID Connect Provider. We generate this in our "init" repo, but you could just as easily place it here and reference it directly.

An example of that infrastructure is:

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = ["6938fd4d98bab03faadb97b34396831e3780aea1"]
}

then replace all the ssm parameter references for oidc_arn with aws_iam_openid_connect_provider.github.arn

Additional Reference

Related Assets

This is a core infrastructure repository that defines infrastructure related to ECS, ECR, and Fargate deployments. The following application infrastructure repositories depend on this repository:

Maintainers

TF markdown is automatically inserted at the bottom of this file, nothing should be written beyond this point

Requirements

Name Version
terraform ~> 1.5
aws ~> 5.0

Providers

Name Version
aws 5.62.0

Modules

Name Source Version
ecr_alma_webhook_lambdas ./modules/ecr n/a
ecr_asati ./modules/ecr n/a
ecr_bursar ./modules/ecr n/a
ecr_carbon ./modules/ecr n/a
ecr_creditcardslips ./modules/ecr n/a
ecr_dss ./modules/ecr n/a
ecr_hrqb_client ./modules/ecr n/a
ecr_matomo ./modules/ecr n/a
ecr_oaiharvester ./modules/ecr n/a
ecr_patronload ./modules/ecr n/a
ecr_ppod ./modules/ecr n/a
ecr_sapinvoices ./modules/ecr n/a
ecr_sapinvoices_ui ./modules/ecr n/a
ecr_timdex_browsertrix ./modules/ecr n/a
ecr_timdex_geo ./modules/ecr n/a
ecr_timdex_lambdas ./modules/ecr n/a
ecr_timdex_tim ./modules/ecr n/a
ecr_timdex_transmogrifier ./modules/ecr n/a
ecr_wcd2reshare ./modules/ecr n/a
ecr_wiley ./modules/ecr n/a

Resources

Name Type
aws_iam_policy.login resource
aws_iam_policy_document.login data source
aws_ssm_parameter.oidc_arn data source

Inputs

Name Description Type Default Required
appinput_ssm_path Standard prefix in Parameter Store for Terraform outputs specifically needed by string n/a yes
aws_region The AWS region where this infrastructure will be deployed. string "us-east-1" no
environment The name of the environment/stage/workspace (e.g., stage, prod, dev) string n/a yes
mitlib The shortname name of our MITLibraries Organization string "mitlib" no
name The name of the app/resource this code builds string n/a yes
ou The name of the AWS OU that contains the AWS Accounts that are managed by this code. string n/a yes
tfinput_ssm_path Standard prefix in Parameter Store for inputs to Terraform for initial infrastructure builds string n/a yes
tfoutput_ssm_path Standard prefix in Parameter Store for generic Terraform outputs (typically needed by other infra code) string n/a yes

Outputs

Name Description
alma_webhook_lambdas_dev_build_workflow Full contents of the dev-build.yml for the alma-webhook-lambdas repo
alma_webhook_lambdas_makefile Full contents of the Makefile for the alma-webhook-lambdas repo (allows devs to push to Dev account only)
alma_webhook_lambdas_prod_promote_workflow Full contents of the prod-promote.yml for the alma-webhook-lambdas repo
alma_webhook_lambdas_stage_build_workflow Full contents of the stage-build.yml for the alma-webhook-lambdas repo
asati_fargate_dev_build_workflow Full contents of the dev-build.yml for the asati repo
asati_fargate_makefile Full contents of the Makefile for the asati repo (allows devs to push to Dev account only)
asati_fargate_prod_promote_workflow Full contents of the prod-promote.yml for the asati repo
asati_fargate_stage_build_workflow Full contents of the stage-build.yml for the asati repo
browsertrix_dev_build_workflow Full contents of the dev-build.yml for the browsertrix-harvester repo
browsertrix_makefile Full contents of the Makefile for the browsertrix-harvester repo (allows devs to push to Dev account only)
browsertrix_prod_promote_workflow Full contents of the prod-promote.yml for the browsertrix-harvester repo
browsertrix_stage_build_workflow Full contents of the stage-build.yml for the browsertrix-harvester repo
bursar_dev_build_workflow Full contents of the dev-build.yml for the alma-bursartransfer repo
bursar_makefile Full contents of the Makefile for the alma-bursartransfer repo (allows devs to push to Dev account only)
bursar_prod_promote_workflow Full contents of the prod-promote.yml for the alma-bursartransfer repo
bursar_stage_build_workflow Full contents of the stage-build.yml for the alma-bursartransfer repo
carbon_dev_build_workflow Full contents of the dev-build.yml for the carbon repo
carbon_makefile Full contents of the Makefile for the carbon repo (allows devs to push to Dev account only)
carbon_prod_promote_workflow Full contents of the prod-promote.yml for the carbon repo
carbon_stage_build_workflow Full contents of the stage-build.yml for the carbon repo
creditcardslips_dev_build_workflow Full contents of the dev-build.yml for the alma-creditcardslips repo
creditcardslips_makefile Full contents of the Makefile for the alma-creditcardslips repo (allows devs to push to Dev account only)
creditcardslips_prod_promote_workflow Full contents of the prod-promote.yml for the alma-creditcardslips repo
creditcardslips_stage_build_workflow Full contents of the stage-build.yml for the alma-creditcardslips repo
dss_fargate_dev_build_workflow Full contents of the dev-build.yml for the dss repo
dss_fargate_makefile Full contents of the Makefile for the dss repo (allows devs to push to Dev account only)
dss_fargate_prod_promote_workflow Full contents of the prod-promote.yml for the dss repo
dss_fargate_stage_build_workflow Full contents of the stage-build.yml for the dss repo
geo_dev_build_workflow Full contents of the dev-build.yml for the geo-harvester repo
geo_makefile Full contents of the Makefile for the geo-harvester repo (allows devs to push to Dev account only)
geo_prod_promote_workflow Full contents of the prod-promote.yml for the geo-harvester repo
geo_stage_build_workflow Full contents of the stage-build.yml for the geo-harvester repo
hrqb_client_fargate_dev_build_workflow Full contents of the dev-build.yml for the hrqb-client repo
hrqb_client_fargate_makefile Full contents of the Makefile for the hrqb-client repo (allows devs to push to Dev account only)
hrqb_client_fargate_prod_promote_workflow Full contents of the prod-promote.yml for the hrqb-client repo
hrqb_client_fargate_stage_build_workflow Full contents of the stage-build.yml for the hrqb-client repo
matomo_fargate_dev_build_workflow Full contents of the dev-build.yml for the matomo repo
matomo_fargate_makefile Full contents of the Makefile for the matomo repo (allows devs to push to Dev account only)
matomo_fargate_prod_promote_workflow Full contents of the prod-promote.yml for the matomo repo
matomo_fargate_stage_build_workflow Full contents of the stage-build.yml for the matomo repo
oaiharvester_dev_build_workflow Full contents of the dev-build.yml for the oaiharvester repo
oaiharvester_makefile Full contents of the Makefile for the oaiharvester repo (allows devs to push to Dev account only)
oaiharvester_prod_promote_workflow Full contents of the prod-promote.yml for the oaiharvester repo
oaiharvester_stage_build_workflow Full contents of the stage-build.yml for the oaiharvester repo
patronload_dev_build_workflow Full contents of the dev-build.yml for the alma-patronload repo
patronload_makefile Full contents of the Makefile for the alma-patronload repo (allows devs to push to Dev account only)
patronload_prod_promote_workflow Full contents of the prod-promote.yml for the alma-patronload repo
patronload_stage_build_workflow Full contents of the stage-build.yml for the alma-patronload repo
ppod_dev_build_workflow Full contents of the dev-build.yml for the ppod repo
ppod_makefile Full contents of the Makefile for the ppod repo (allows devs to push to Dev account only)
ppod_prod_promote_workflow Full contents of the prod-promote.yml for the ppod repo
ppod_stage_build_workflow Full contents of the stage-build.yml for the ppod repo
sapinvoices_dev_build_workflow Full contents of the dev-build.yml for the alma-sapinvoices repo
sapinvoices_makefile Full contents of the Makefile for the alma-sapinvoices repo (allows devs to push to Dev account only)
sapinvoices_prod_promote_workflow Full contents of the prod-promote.yml for the alma-sapinvoices repo
sapinvoices_stage_build_workflow Full contents of the stage-build.yml for the alma-sapinvoices repo
sapinvoices_ui_dev_build_workflow Full contents of the dev-build.yml for the alma-sapinvoices-ui repo
sapinvoices_ui_makefile Full contents of the Makefile for the alma-sapinvoices-ui repo (allows devs to push to Dev account only)
sapinvoices_ui_prod_promote_workflow Full contents of the prod-promote.yml for the alma-sapinvoices-ui repo
sapinvoices_ui_stage_build_workflow Full contents of the stage-build.yml for the alma-sapinvoices-ui repo
tim_dev_build_workflow Full contents of the dev-build.yml for the timdex-index-manager repo
tim_makefile Full contents of the Makefile for the timdex-index-manager repo (allows devs to push to Dev account only)
tim_prod_promote_workflow Full contents of the prod-promote.yml for the timdex-index-manager repo
tim_stage_build_workflow Full contents of the stage-build.yml for the timdex-index-manager repo
timdex_lambdas_dev_build_workflow Full contents of the dev-build.yml for the timdex-pipeline-lambdas repo
timdex_lambdas_makefile Full contents of the Makefile for the timdex-pipeline-lambdas repo (allows devs to push to Dev account only)
timdex_lambdas_prod_promote_workflow Full contents of the prod-promote.yml for the timdex-pipeline-lambdas repo
timdex_lambdas_stage_build_workflow Full contents of the stage-build.yml for the timdex-pipeline-lambdas repo
transmogrifier_dev_build_workflow Full contents of the dev-build.yml for the transmogrifier repo
transmogrifier_makefile Full contents of the Makefile for the transmogrifier repo (allows devs to push to Dev account only)
transmogrifier_prod_promote_workflow Full contents of the prod-promote.yml for the transmogrifier repo
transmogrifier_stage_build_workflow Full contents of the stage-build.yml for the transmogrifier repo
wcd2reshare_dev_build_workflow Full contents of the dev-build.yml for the wcd2reshare repo
wcd2reshare_makefile Full contents of the Makefile for the wcd2reshare repo (allows devs to push to Dev account only)
wcd2reshare_prod_promote_workflow Full contents of the prod-promote.yml for the wcd2reshare repo
wcd2reshare_stage_build_workflow Full contents of the stage-build.yml for the wcd2reshare repo
wiley_fargate_dev_build_workflow Full contents of the dev-build.yml for the wiley-deposits repo
wiley_fargate_makefile Full contents of the Makefile for the wiley-deposits repo (allows devs to push to Dev account only)
wiley_fargate_prod_promote_workflow Full contents of the prod-promote.yml for the wiley-deposits repo
wiley_fargate_stage_build_workflow Full contents of the stage-build.yml for the wiley-deposits repo

About

ECR repos, lifecycle policies, and their associated OIDC authentication for github repos publishing to them

Resources

License

Stars

Watchers

Forks

Packages

No packages published