Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move test helpers back to digital ocean #91

Closed
Tracked by #92
hellais opened this issue Aug 19, 2024 · 2 comments
Closed
Tracked by #92

Move test helpers back to digital ocean #91

hellais opened this issue Aug 19, 2024 · 2 comments

Comments

@hellais
Copy link
Member

hellais commented Aug 19, 2024

Currently because of NAT gateways it's costing us a lot to run the test helpers on AWS, we should move them to digital ocean.

@hellais
Copy link
Member Author

hellais commented Sep 6, 2024

Some work related to this done here: #96

@hellais
Copy link
Member Author

hellais commented Sep 24, 2024

Checklist:

  • Deploy 3 new test helpers on digital ocean without updating records
  • Update 1 record of new digital ocean test helper and monitor traffic
  • Update remaining 2 records of new digital ocean test helpers
  • Cleanup all test helper related hosts

hellais added a commit that referenced this issue Oct 1, 2024
Changes:
* Add support for creating digital ocean droplets for test helpers
* Switch networking config to NAT gateway less setup

Fixes:
* #92
* #91
* #93

Checklist for doing it:
* [x] Create terraform module for deploying test helpers to digital
ocean
* [x] Tweak ECS task sizes to reduce instance consumption
* [x] Drop ECS cluster for test helpers
* [x] Setup direct load balancer rules to address services based on
hostname
* [x] Drop test helper monitoring on AWS in monitoring host
* [x] Add support for SAN in ACM certificates by creating new module
* [x] Setup nginx based load balancing for test helpers
* [x] Refactor EC2 instance deployment to use cloud-init

Since AWS costs are too high for the test helpers with IPv6 support, we
switch them back to digital ocean.

How this is implemented is by adding a new rule to the oonibackend proxy
that acts as a load balancer towards the test helpers on digital ocean.
The reason to do this is so that we don't have to complicate the TLS
setup by having to do certificate provisioning on the test helpers, but
rather are able to keep it in AWS.

Moreover by having a single entry point to the test helpers it means we
can implement a cache which works across all the test helper backends,
instead of having a per-test helper caching layer.

What is missing is adding the rules that perform routing on a per domain
basis to the load balancer config.

---------

Co-authored-by: decfox <[email protected]>
@DecFox DecFox closed this as completed Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants