Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate dev and prod deployments and switch th back to DO #96

Merged
merged 66 commits into from
Oct 1, 2024

Conversation

hellais
Copy link
Member

@hellais hellais commented Sep 5, 2024

Changes:

  • Add support for creating digital ocean droplets for test helpers
  • Switch networking config to NAT gateway less setup

Fixes:

Checklist for doing it:

  • Create terraform module for deploying test helpers to digital ocean
  • Tweak ECS task sizes to reduce instance consumption
  • Drop ECS cluster for test helpers
  • Setup direct load balancer rules to address services based on hostname
  • Drop test helper monitoring on AWS in monitoring host
  • Add support for SAN in ACM certificates by creating new module
  • Setup nginx based load balancing for test helpers
  • Refactor EC2 instance deployment to use cloud-init

Since AWS costs are too high for the test helpers with IPv6 support, we switch them back to digital ocean.

How this is implemented is by adding a new rule to the oonibackend proxy that acts as a load balancer towards the test helpers on digital ocean. The reason to do this is so that we don't have to complicate the TLS setup by having to do certificate provisioning on the test helpers, but rather are able to keep it in AWS.

Moreover by having a single entry point to the test helpers it means we can implement a cache which works across all the test helper backends, instead of having a per-test helper caching layer.

What is missing is adding the rules that perform routing on a per domain basis to the load balancer config.

Copy link

github-actions bot commented Sep 5, 2024

Terraform Run Output 🤖

Format and Style 🖌failure

Initialization ⚙️success

Validation 🤖success

Validation Output

$ terraform validate
Success! The configuration is valid.

Plan 📖success

  • Plan: 0 to add, 2 to change, 0 to destroy.
Show Plan

$ terraform plan
Acquiring state lock. This may take a few moments...
module.ansible_inventory.local_file.ansible_inventory: Refreshing state... [id=b6de844ed8d384f890fa6f467502390de843f758]
random_password.jwt_secret: Refreshing state... [id=none]
random_password.prometheus_metrics_password: Refreshing state... [id=none]
random_id.artifact_id: Refreshing state... [id=8Ujqew]
module.adm_iam_roles.tls_private_key.oonidevops: Refreshing state... [id=b49a9fdb9f720320340226016efe24808dd68203]
module.ansible_inventory.null_resource.ansible_update_known_hosts: Refreshing state... [id=236461505953331670]
module.ooniapi_user.aws_ses_email_identity.ooniapi: Refreshing state... [[email protected]]
module.ooniapi_ooniauth.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniauth]
data.aws_availability_zones.available: Reading...
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
aws_secretsmanager_secret.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw]
module.ooniapi_ooniprobe.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role]
module.ooniapi_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/ooniapi-ecs-cluster]
aws_secretsmanager_secret.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Reading...
aws_acm_certificate.ooniapi_frontend: Refreshing state... [id=arn:aws:acm:eu-central-1:905418398257:certificate/190205f1-392d-425c-a059-7006ca8c8c46]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_oonirun_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonirun]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooniapi_cluster.aws_iam_role.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonifindings-td:12/ooniapi-service-oonifindings]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Reading...
module.ooni_backendproxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Reading...
data.aws_availability_zones.available: Read complete after 0s [id=eu-central-1]
module.ooniapi_oonifindings.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.adm_iam_roles.aws_key_pair.oonidevops: Refreshing state... [id=oonidevops]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td:37/ooniapi-service-ooniprobe]
module.ooniapi_oonirun.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonirun]
module.ooniapi_user.aws_iam_user.ooniapi: Refreshing state... [id=oonidevops-ooniapi]
module.ooniapi_user.aws_secretsmanager_secret.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Reading...
module.oonidevops_github_user.aws_iam_policy.oonidevops_github: Refreshing state... [id=arn:aws:iam::905418398257:policy/oonidevops-github-policy]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_oonirun.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role]
module.ooniapi_user.aws_secretsmanager_secret.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx]
module.ooniapi_oonifindings.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonifindings]
module.adm_iam_roles.aws_iam_policy.oonidevops: Refreshing state... [id=arn:aws:iam::905418398257:policy/OONIDevopsPolicy]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_ooniprobe.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniprobe]
module.adm_iam_roles.aws_secretsmanager_secret.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
aws_secretsmanager_secret.jwt_secret: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/jwt_secret-NUESvS]
module.oonidevops_github_user.aws_iam_user.oonidevops_github: Refreshing state... [id=oonidevops-github]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonifindings]
module.oonidevops_github_user.aws_secretsmanager_secret.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd]
aws_s3_bucket.ooniapi_codepipeline_bucket: Refreshing state... [id=codepipeline-ooniapi-eu-central-1-f148ea7b]
aws_s3_bucket.oonith_codepipeline_bucket: Refreshing state... [id=codepipeline-oonith-eu-central-1-f148ea7b]
module.ooni_backendproxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 0s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniauth]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniprobe]
module.ooniapi_ooniauth.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role]
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Reading...
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Read complete after 0s [id=4022892340]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonirun]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 1s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonirun-td:54/ooniapi-service-oonirun]
module.ooniapi_ooniprobe.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role:ooniapi-service-ooniprobe-task-role]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Reading...
aws_secretsmanager_secret_version.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|terraform-20240314200140936700000008]
module.ooniapi_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=ooniapi-ecs-cluster]
module.ooniapi_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:ooniapi-ecs-cluster-instance-role-policy]
module.ooniapi_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/ooniapi-ecs-cluster]
module.ooniapi_user.aws_iam_user_policy.ooniapi: Refreshing state... [id=oonidevops-ooniapi:oonidevops-ooniapi-policy]
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniauth-td:59/ooniapi-service-ooniauth]
module.ooniapi_user.aws_iam_access_key.ooniapi: Refreshing state... [id=AKIA5FTZELIYSK2XEVOT]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|AWSCURRENT]
aws_route53_record.ooniapi_frontend_cert_validation["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__48cd4e71cee9930614228176b7deefb9.ooniauth.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["api.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__cd4729fc0c282e771d056e719a7bdf4f.api.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__ef17825e5fd9713f596344bdd9626f5e.8.th.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__05c891caeb4509d4cd7f9c24d8b6dbd0.oonirun.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__a064be8aa084a037ff9fa5e3e541c87d.ooniprobe.dev.ooni.io._CNAME]
module.ooniapi_oonifindings.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role:ooniapi-service-oonifindings-task-role]
module.oonidevops_github_user.aws_iam_user_policy_attachment.oonidevops_github: Refreshing state... [id=oonidevops-github-20240313195612421500000001]
module.oonidevops_github_user.aws_iam_access_key.oonidevops_github: Refreshing state... [id=AKIA5FTZELIYXDN55SMS]
module.ooniapi_oonirun.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role:ooniapi-service-oonirun-task-role]
module.ooniapi_oonifindings_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonifindings]
module.ooniapi_ooniauth_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniauth]
module.adm_iam_roles.aws_iam_role.oonidevops: Refreshing state... [id=oonidevops]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniprobe]
module.adm_iam_roles.aws_secretsmanager_secret_version.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|terraform-20240925140131946100000002]
aws_secretsmanager_secret_version.jwt_secret: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/jwt_secret-NUESvS|terraform-20240310182536838400000005]
module.ooniapi_ooniauth.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role:ooniapi-service-ooniauth-task-role]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr|terraform-20240314200140914600000006]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx|terraform-20240314200140918400000007]
module.oonidevops_github_user.aws_secretsmanager_secret_version.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd|terraform-20240519071250187000000004]
aws_acm_certificate_validation.ooniapi_frontend: Refreshing state... [id=0001-01-01 00:00:00 +0000 UTC]
data.aws_secretsmanager_secret_version.deploy_key: Reading...
aws_codestarconnections_connection.oonidevops: Refreshing state... [id=arn:aws:codestar-connections:eu-central-1:905418398257:connection/6bd492f6-c11d-43ec-92b0-24c47700d528]
module.oonipg.random_password.pg_password: Refreshing state... [id=none]
module.network.aws_vpc.main: Refreshing state... [id=vpc-0e382f3ad89286de9]
module.terraform_state_backend.data.aws_region.current: Reading...
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
module.oonipg.aws_secretsmanager_secret.pg_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/pg_password-OjzOJC]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=oonidevops-dev-terraform-state-lock]
data.aws_secretsmanager_secret_version.deploy_key: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|AWSCURRENT]
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Reading...
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Read complete after 0s [id=2419757651]
module.ooni_th_droplet.digitalocean_droplet.ooni_th_docker[0]: Refreshing state... [id=447478398]
module.oonipg.aws_secretsmanager_secret_version.pg_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/pg_password-OjzOJC|terraform-20240310155428358300000002]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniprobe-eu-central-1]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniauth-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonirun-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonifindings-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonirun]
module.ooniapi_oonifindings_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonifindings]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniprobe]
module.ooniapi_ooniauth_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniprobe]
module.ooniapi_oonifindings_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonifindings]
module.ooniapi_oonirun_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonirun]
module.ooniapi_ooniauth_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniauth]
module.network.aws_internet_gateway.gw: Refreshing state... [id=igw-0c080e9b235ed29d1]
module.ooniapi_cluster.aws_security_group.web: Refreshing state... [id=sg-0187eedfe39538357]
module.ooniapi_oonirun.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrunM-20240917211808753100000003/93d4b2b6dc76acac]
module.ooniapi_ooniauth.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OautM-20240917211808752700000001/91bcad756924a3a7]
module.ooniapi_oonifindings.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OfinM-20240917211808752800000002/48adcfb18ae34d30]
module.oonipg.aws_security_group.pg: Refreshing state... [id=sg-005ca579eb9c08cda]
module.ooniapi_ooniprobe.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OproM-20240917211808753100000004/1f2ea1732205872c]
module.ooni_backendproxy.aws_alb_target_group.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oobpx20240422160053180600000001/1ec17e88e2467a5b]
module.network.aws_route_table.public: Refreshing state... [id=rtb-0ccb0852e6a365a95]
module.network.aws_route_table.private: Refreshing state... [id=rtb-011463437da96c77b]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-09314a43ec89d6331]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0b899a7ad10406d06]
module.network.aws_subnet.public[1]: Refreshing state... [id=subnet-0b18966cccfc9d5ef]
module.network.aws_subnet.public[0]: Refreshing state... [id=subnet-0e7a4478be988463f]
module.ooniapi_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0aa6a97400b619de3]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.network.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-0e7933e6b804ff2c1]
module.network.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0c9cc0f117ef15fe7]
module.ooni_backendproxy.aws_security_group.nginx_sg: Refreshing state... [id=sg-050fe64c8d862ab5e]
module.network.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-08ab18165bf481054]
module.network.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-0dbd7fb16801ee049]
module.oonipg.aws_db_subnet_group.pg: Refreshing state... [id=ooni-tier0-postgres-dbsng]
module.ooniapi_frontend.aws_alb.ooniapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:loadbalancer/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6]
module.ooniapi_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0e328a8671f870c64]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-03-10T15:06:17Z]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooniapi_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=ooniapi-ecs-cluster20240310192644083800000003]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_http: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/d9b2448464179cd1]
aws_route53_record.ooniapi_frontend_main: Refreshing state... [id=Z055356431RGCLK3JXZDL_api.dev.ooni.io_A]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd]
aws_route53_record.ooniapi_frontend_alt["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonirun.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_8.th.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniauth.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniprobe.dev.ooni.io_A]
module.oonipg.aws_db_instance.pg: Refreshing state... [id=db-27N7Q6XIBNASFCOXN4N7C762L4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/65e6f5e3aca0a4e5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/2b09ed268181ba4f]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/551c4128bb282fa4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cdc4e8e8eabb56f2]
module.ooniapi_frontend.aws_alb_listener_rule.ooniapi_th: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/01ee7503374be8ff]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/b436b91883ae7c86]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cefeff0d8aa3118a]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/5a872da4cc6b9135]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/128c53ea760208fc]
aws_route53_record.postgres_dns: Refreshing state... [id=Z091407123AEJO90Z3H6D_postgres.dev.ooni.nu_CNAME]
aws_secretsmanager_secret_version.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ|terraform-20240310182536837800000004]
module.ooniapi_oonifindings.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonifindings-td]
module.ooniapi_ooniauth.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniauth-td]
module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniprobe-td]
module.ooniapi_oonirun.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonirun-td]
module.ooniapi_oonifindings.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonifindings]
module.ooniapi_ooniauth.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniauth]
module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.ooniapi_oonirun.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonirun]
module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonifindings]
module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniprobe]
module.ooniapi_oonirun_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonirun]
module.ooniapi_ooniauth_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniauth]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-oonifindings"
        name           = "ooniapi-oonifindings"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "master",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

  # module.oonipg.aws_security_group.pg will be updated in-place
  ~ resource "aws_security_group" "pg" {
      ~ egress                 = [
          - {
              - cidr_blocks      = [
                  - "0.0.0.0/0",
                ]
              - from_port        = 0
              - ipv6_cidr_blocks = []
              - prefix_list_ids  = []
              - protocol         = "-1"
              - security_groups  = []
              - self             = false
              - to_port          = 0
                # (1 unchanged attribute hidden)
            },
          - {
              - cidr_blocks      = []
              - from_port        = 0
              - ipv6_cidr_blocks = [
                  - "::/0",
                ]
              - prefix_list_ids  = []
              - protocol         = "-1"
              - security_groups  = []
              - self             = false
              - to_port          = 0
                # (1 unchanged attribute hidden)
            },
          + {
              + cidr_blocks      = [
                  + "0.0.0.0/0",
                ]
              + from_port        = 0
              + ipv6_cidr_blocks = []
              + prefix_list_ids  = []
              + protocol         = "-1"
              + security_groups  = []
              + self             = false
              + to_port          = 0
            },
        ]
        id                     = "sg-005ca579eb9c08cda"
        name                   = "oonipg20240422154923257500000004"
        tags                   = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-postgres"
            "Repository"  = "https://github.com/ooni/devops"
        }
        # (8 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.
Pusher @hellais
Action pull_request
Environment dev
Workflow .github/workflows/check_terraform.yml
Last updated Tue, 01 Oct 2024 14:43:25 GMT

Copy link

github-actions bot commented Sep 17, 2024

Ansible Run Output 🤖

Ansible Playbook Recap 🔍



Ansible playbook output 📖success

Show Execution

$ ansible-playbook playbook.yml --check --diff -i ../tf/modules/ansible_inventory/inventories/inventory-dev.ini
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
[WARNING]: Could not match supplied host pattern, ignoring: clickhouse_servers
[WARNING]: Could not match supplied host pattern, ignoring: monitoring.ooni.org
[WARNING]: Could not match supplied host pattern, ignoring: data.ooni.org
[WARNING]: Could not match supplied host pattern, ignoring: openvpn-
server1.ooni.io

PLAY [ClickHouse servers] ******************************************************
skipping: no hosts matched

PLAY [Update monitoring config] ************************************************
skipping: no hosts matched

PLAY [Deploy data.ooni.org host] ***********************************************
skipping: no hosts matched

PLAY [Setup OpenVPN server] ****************************************************
skipping: no hosts matched

PLAY RECAP *********************************************************************

Pusher @hellais
Action pull_request
Working Directory
Workflow .github/workflows/check_ansible.yml
Last updated Tue, 01 Oct 2024 14:43:57 GMT

@hellais hellais changed the title Add support for creating digital ocean droplets for test helpers Consolidate dev and prod deployments and switch th back to DO Oct 1, 2024
@hellais
Copy link
Member Author

hellais commented Oct 1, 2024

Related: #62

@hellais hellais merged commit 4876b9e into main Oct 1, 2024
3 checks passed
@hellais hellais deleted the digital-ocean-th branch October 1, 2024 14:46
hellais added a commit that referenced this pull request Oct 1, 2024
* 'main' of github.com:ooni/devops:
  Consolidate dev and prod deployments and switch th back to DO (#96)
hellais added a commit that referenced this pull request Oct 1, 2024
* main:
  Consolidate dev and prod deployments and switch th back to DO (#96)
  Add vasilis to user list
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants