-
Notifications
You must be signed in to change notification settings - Fork 215
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
test(mc): initial multicloud IaC boilerplate GKE, AKS, Kind (#1268)
# Description Initial work on multi-cloud * create modules for aks, gke, kind and retina * use gcloud, azure, kind and helm providers * create examples on how to use each module * create automated tests in Go Terratest for each module example (only initial work, since we require creds for public clouds) * create integration test retina-kind (this can also be used as an example for AKS, GKE and eventually EKS) * create a Makefile for this subproject * create GH workflow with automated tests ![diagram](https://github.com/user-attachments/assets/38ded8ec-0c31-4e5d-9bd0-8729115cf5b3) ## Related Issue #1267 ## Checklist - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [x] I have added tests, if applicable. ## Screenshots (if applicable) or Testing Completed All modules were tested and work as expected. ![Screenshot 2025-01-24 at 12 18 55](https://github.com/user-attachments/assets/2d222a13-2b12-4a11-a001-1b03a254e152) See automated tests here https://github.com/microsoft/retina/actions/runs/12993893617/job/36237118572 ![image](https://github.com/user-attachments/assets/edafa98b-1d5d-44b0-a393-1bf7b7b70bda) ## Additional Notes Add any additional notes or context about the pull request here. --- Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more information on how to contribute to this project.
- Loading branch information
Showing
62 changed files
with
5,087 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
name: Test Multicloud | ||
|
||
on: | ||
pull_request: | ||
paths: | ||
- 'test/multicloud/**' | ||
|
||
jobs: | ||
multicloud-test: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: opentofu/setup-opentofu@v1 | ||
with: | ||
tofu_version: 1.8.3 | ||
|
||
- name: Checkout code | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up Go | ||
uses: actions/setup-go@v2 | ||
with: | ||
go-version: '1.23' | ||
|
||
- name: Install dependencies | ||
run: go mod download | ||
working-directory: test/multicloud/ | ||
|
||
- name: Run tests | ||
run: make test | ||
working-directory: test/multicloud/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
terraform.tfvars | ||
.terraform | ||
terraform.tfstate | ||
*terraform.tfstate.* | ||
service-key.json | ||
*-kind-config |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
PREFIX ?= retina | ||
STACK_NAME ?= $(PREFIX)-aks | ||
|
||
.PHONY: init plan apply quick gke aks kind destroy clean kind-kubeconfig test | ||
|
||
plan: | ||
cd live/$(STACK_NAME) && \ | ||
tofu fmt && tofu init && tofu plan | ||
|
||
apply: | ||
cd live/$(STACK_NAME) && \ | ||
tofu apply --auto-approve | ||
|
||
quick: | ||
@make plan | ||
@make apply | ||
|
||
gke: export STACK_NAME=$(PREFIX)-gke | ||
gke: | ||
@make quick | ||
|
||
aks: export STACK_NAME=$(PREFIX)-aks | ||
aks: | ||
@make quick | ||
|
||
kind: export STACK_NAME=$(PREFIX)-kind | ||
kind: | ||
@make quick | ||
|
||
destroy: | ||
cd live/$(STACK_NAME) && \ | ||
tofu destroy --auto-approve | ||
|
||
clean: destroy | ||
@cd live/$(STACK_NAME) && \ | ||
rm -rf .terraform && rm terraform.tfstate && rm terraform.tfstate.backup | ||
|
||
kind-kubeconfig: | ||
@kubectl config set-context live/$(PREFIX)-kind/mc-kind-config | ||
|
||
test: | ||
@cd test && go test -v -count=1 -timeout 30m ./... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
# Multi Cloud Retina | ||
|
||
This project leverages [OpenTofu](https://opentofu.org/docs/intro/) Infrastructure as Code (IaC) to create Kubernetes infrastructure on multi-cloud and deploy [microsoft/retina](https://github.com/microsoft/retina) via Helm provider. | ||
|
||
![Architecture Diagram](./diagrams/diagram.svg) | ||
|
||
## Modules available | ||
|
||
* [aks](./modules/aks/) | ||
* [gke](./modules/gke/) | ||
* [kind](./modules/kind/) | ||
* [retina](./modules/retina/) | ||
|
||
## Prerequisites | ||
|
||
* [OpenTofu installation guide](https://opentofu.org/docs/intro/install/) | ||
|
||
* AKS: | ||
|
||
1. create an Azure account | ||
2. [Install az](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) | ||
|
||
To deploy an AKS cluster and install retina, create file `live/retina-aks/terraform.tfvars` with the Azure TenantID and SubscriptionID | ||
|
||
```sh | ||
# example values | ||
subscription_id = "d6050d84-e4dd-463d-afc7-a6ab3dc33ab7" | ||
tenant_id = "ac8a4ccd-35f1-4f95-a688-f68e3d89adfc" | ||
``` | ||
|
||
* GKE: | ||
|
||
1. create a gcloud account, project and enable billing | ||
2. create a service account and service account key | ||
3. [Enable Kubernetes Engine API](https://console.developers.google.com/apis/api/container.googleapis.com/overview?project=mc-retina) | ||
4. [Install gcloud](https://cloud.google.com/sdk/docs/install) | ||
|
||
To deploy a GKE cluster export `GOOGLE_APPLICATION_CREDENTIALS` env variable to point to the path where your [service account key](https://cloud.google.com/iam/docs/keys-create-delete) is located. | ||
|
||
```sh | ||
# example | ||
export GOOGLE_APPLICATION_CREDENTIALS=/Users/srodi/src/retina/test/multicloud/live/retina-gke/service-key.json | ||
``` | ||
|
||
* Kind: | ||
|
||
1. Docker installed on the host machine | ||
|
||
## Quickstart | ||
|
||
The following Make targets can be used to manage each stack lifecycle. | ||
|
||
### Create | ||
|
||
Format code, initialize OpenTofu, plan and apply the stack to create infra and deploy retina | ||
|
||
* AKS: | ||
|
||
```sh | ||
make aks | ||
``` | ||
|
||
* GKE: | ||
|
||
```sh | ||
make gke | ||
``` | ||
|
||
* Kind: | ||
|
||
```sh | ||
make kind | ||
``` | ||
|
||
### Clean up | ||
|
||
To destroy the cluster specify the `STACK_NAME` and run `make clean`. | ||
|
||
```sh | ||
# destroy AKS and cleanup local state files | ||
# set a different stack as needed (i.e. retina-gke, retina-kind) | ||
export STACK_NAME=retina-aks | ||
make clean | ||
``` | ||
|
||
### Test | ||
|
||
The test framework is levergaing Go and [Terratest](https://terratest.gruntwork.io/docs/). To run tests: | ||
|
||
```sh | ||
make test | ||
``` | ||
|
||
## Providers references | ||
|
||
* [GKE resource documentation](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster) | ||
* [AKS resource documentation](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster) | ||
* [Kind resource documentation](https://registry.terraform.io/providers/tehcyx/kind/latest/docs/resources/cluster) |
Oops, something went wrong.