superset-on-azure

Prerequisites

This solution was developed/tested using the following:

• Terraform (v1.0.6)
• Docker (v20.10.8)
• Azure CLI (v2.28.0)
• kubectl (v1.20.2)
• envsubst (v0.19.8.1)
• jq (v1.6)

Infrastructure

The following commands should get your infrastructure set up.

Good luck! 🤞

1. Perform an interactive login to Azure using the CLI:

   az login

   This should open your preferred browser with a prompt to log in to your account.

2. Now you have authenticated to Azure you can get cracking with the Terraform:

   terraform init

3. Now let's ask Terraform to provision the various resources.

   terraform apply

   Sanity check the output and respond appropriately (most likely with a yes, 'cause you want your Superset resources, right?)

4. Go make yourself a brew ☕.

   The provisioning might take a while (the Redis instance seems to be the culprit responsible for the significant wait time at a whopping 17 minutes last time I tried. Bonkers).

Superset Container

1. Get the acr_login_server value from Terraform and assign it to an environment variable:

   export ACR_LOGIN_SERVER=$(terraform output -raw acr_login_server)

2. Have a quick sanity check of the variable value:

   echo $ACR_LOGIN_SERVER

3. Log in to the Azure Container Registry (ACR):

   az acr login --name $ACR_LOGIN_SERVER

4. Build the Superset Docker container:

   docker build --tag $ACR_LOGIN_SERVER/superset_base .

5. Push the container to the ACR:

   docker push $ACR_LOGIN_SERVER/superset_base

Deploy Superset to Your Azure Kubernetes Service (AKS)

Before we begin it is important to remind you, dear reader, that this whole thing is a demonstration on how to deploy Superset to Azure. It is not illustrating best practices, and a prime example of this is shown in step 1 below where we will be grabbing a secret from a secure KeyVault, and assigning it to an environment variable. Hmmm...

1. Set up the environment variables:

   export RESOURCE_GROUP_NAME=$(terraform output -raw resource_group_name)
   export AKS_CLUSTER_NAME=$(terraform output -raw aks_cluster_name)
   export ACR_LOGIN_SERVER=$(terraform output -raw acr_login_server)
   export SUPERSET_WEB_IP=$(terraform output -raw superset_web_ip)
   export DATABASE_HOST=$(terraform output -raw databse_host)
   export DATABASE_ADMIN_USERNAME=$(terraform output -raw database_admin_username)
   export DATABASE_ADMIN_PASSWORD_SECRET_ID=$(terraform output -raw database_admin_password_secret_id)
   export DATABASE_ADMIN_PASSWORD=$(az keyvault secret show --id $DATABASE_ADMIN_PASSWORD_SECRET_ID | jq -r .value)
   export REDIS_HOST=$(terraform output -raw redis_host)
   export REDIS_PORT=$(terraform output -raw redis_port)

2. Connect to the AKS cluster:

   az aks get-credentials --resource-group $RESOURCE_GROUP_NAME --name $AKS_CLUSTER_NAME

3. Create the namespace:

   kubectl apply --filename kubernetes/namespace.yaml

4. Set up the services:

   cat kubernetes/services.yaml | envsubst | kubectl apply --filename -

   Note: this uses envsubst to perform token replacement within the *.yaml files before piping them in to kubectl via stdin.

5. Set up the secrets:

   cat kubernetes/secrets.yaml | envsubst | kubectl apply --filename -

6. Set up the config map (environment variables):

   cat kubernetes/config.yaml | envsubst | kubectl apply --filename -

7. Set up the initialisation job (this performs tasks like the initial database object creation):

   cat kubernetes/job.yaml | envsubst | kubectl apply --filename -

8. Set up the superset application:

   cat kubernetes/deployment.yaml | envsubst | kubectl apply --filename -

9. Get the URL for the Superset Application:

   echo http://$SUPERSET_WEB_IP

10. Go have fun on with your new Superset instance! 🎉