In this lab we will create our Azure Kubernetes Services (AKS) distributed compute cluster.
- Azure Account
-
Login to Azure Portal at http://portal.azure.com.
-
Open the Azure Cloud Shell and choose Bash Shell (do not choose Powershell)
-
The first time Cloud Shell is started will require you to create a storage account.
-
Once your cloud shell is started, clone the workshop repo into the cloud shell environment
git clone https://github.com/Azure/kubernetes-hackfest cd kubernetes-hackfest/labs/create-aks-cluster
Note: In the cloud shell, you are automatically logged into your Azure subscription.
-
Ensure you are using the correct Azure subscription you want to deploy AKS to.
# View subscriptions az account list
# Verify selected subscription az account show
# Set correct subscription (if needed) az account set --subscription <subscription_id> # Verify correct subscription is now set az account show
-
Create Azure Service Prinicpal to use through the labs
az ad sp create-for-rbac --skip-assignment
This will return the following. !!!IMPORTANT!!! - Please copy this information down as you'll need it for labs going forward.
"appId": "7248f250-0000-0000-0000-dbdeb8400d85", "displayName": "azure-cli-2017-10-15-02-20-15", "name": "http://azure-cli-2017-10-15-02-20-15", "password": "77851d2c-0000-0000-0000-cb3ebc97975a", "tenant": "72f988bf-0000-0000-0000-2d7cd011db47"
Set the values from above as variables (replace and with your values).
DON'T MESS THIS STEP UP. REPLACE THE VALUES IN BRACKETS!!!
# Persist for Later Sessions in Case of Timeout APPID=<appId> echo export APPID=$APPID >> ~/.bashrc CLIENTSECRET=<password> echo export CLIENTSECRET=$CLIENTSECRET >> ~/.bashrc
-
Create a unique identifier suffix for resources to be created in this lab.
UNIQUE_SUFFIX=$USER$RANDOM # Remove Underscores and Dashes (Not Allowed in AKS and ACR Names) UNIQUE_SUFFIX="${UNIQUE_SUFFIX//_}" UNIQUE_SUFFIX="${UNIQUE_SUFFIX//-}" # Check Unique Suffix Value (Should be No Underscores or Dashes) echo $UNIQUE_SUFFIX # Persist for Later Sessions in Case of Timeout echo export UNIQUE_SUFFIX=$UNIQUE_SUFFIX >> ~/.bashrc
*** Note this value as it will be used in the next couple labs. ***
-
Create an Azure Resource Group in East US.
# Set Resource Group Name RGNAME=kubernetes-hackfest # Persist for Later Sessions in Case of Timeout echo export RGNAME=kubernetes-hackfest >> ~/.bashrc # Set Region (Location) LOCATION=eastus # Persist for Later Sessions in Case of Timeout echo export LOCATION=eastus >> ~/.bashrc # Create Resource Group az group create -n $RGNAME -l $LOCATION
-
Create your AKS cluster in the resource group created above with 3 nodes, targeting Kubernetes version 1.10.3, with Container Insights, and HTTP Application Routing Enabled. You will use the Service Principal information from step 5.
Use Unique CLUSTERNAME
# Set AKS Cluster Name CLUSTERNAME=aks${UNIQUE_SUFFIX} # Look at AKS Cluster Name for Future Reference echo $CLUSTERNAME # Persist for Later Sessions in Case of Timeout echo export CLUSTERNAME=aks${UNIQUE_SUFFIX} >> ~/.bashrc
The below command can take 10-20 minutes to run as it is creating the AKS cluster. Please be PATIENT and grab a coffee...
# Create AKS Cluster az aks create -n $CLUSTERNAME -g $RGNAME -k 1.11.4 \ --service-principal $APPID \ --client-secret $CLIENTSECRET \ --generate-ssh-keys -l $LOCATION \ --node-count 3 \ --enable-addons http_application_routing,monitoring
-
Verify your cluster status. The
ProvisioningState
should beSucceeded
az aks list -o table
Name Location ResourceGroup KubernetesVersion ProvisioningState Fqdn ------------------- ---------- -------------------- ------------------- ------------------- ------------------------------------------------------------------- ODLaks-v2-gbb-16502 eastus ODL_aks-v2-gbb-16502 1.11.4 Succeeded odlaks-v2--odlaks-v2-gbb-16-b23acc-17863579.hcp.centralus.azmk8s.io
-
Get the Kubernetes config files for your new AKS cluster
az aks get-credentials -n $CLUSTERNAME -g $RGNAME
-
Verify you have API access to your new AKS cluster
Note: It can take 5 minutes for your nodes to appear and be in READY state. You can run
watch kubectl get nodes
to monitor status.kubectl get nodes
NAME STATUS ROLES AGE VERSION aks-nodepool1-26522970-0 Ready agent 33m v1.11.3
To see more details about your cluster:
kubectl cluster-info
Kubernetes master is running at https://cluster-dw-kubernetes-hackf-80066e-a44f3eb0.hcp.eastus.azmk8s.io:443 addon-http-application-routing-default-http-backend is running at https://cluster-dw-kubernetes-hackf-80066e-a44f3eb0.hcp.eastus.azmk8s.io:443/api/v1/namespaces/kube-system/services/addon-http-application-routing-default-http-backend/proxy addon-http-application-routing-nginx-ingress is running at http://168.62.191.18:80 http://168.62.191.18:443 Heapster is running at https://cluster-dw-kubernetes-hackf-80066e-a44f3eb0.hcp.eastus.azmk8s.io:443/api/v1/namespaces/kube-system/services/heapster/proxy KubeDNS is running at https://cluster-dw-kubernetes-hackf-80066e-a44f3eb0.hcp.eastus.azmk8s.io:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy kubernetes-dashboard is running at https://cluster-dw-kubernetes-hackf-80066e-a44f3eb0.hcp.eastus.azmk8s.io:443/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy
You should now have a Kubernetes cluster running with 3 nodes. You do not see the master servers for the cluster because these are managed by Microsoft. The Control Plane services which manage the Kubernetes cluster such as scheduling, API access, configuration data store and object controllers are all provided as services to the nodes.
This lab creates namespaces that reflect a representative example of an organization's environments. In this case DEV, UAT and PROD. We will also apply the appopriate permissions, limits and resource quotas to each of the namespaces.
- Build AKS Cluster (from above)
-
Create Three Namespaces
# Create namespaces kubectl apply -f create-namespaces.yaml # Look at namespaces kubectl get ns
-
Assign CPU, Memory and Storage Limits to Namespaces
# Create namespace limits kubectl apply -f namespace-limitranges.yaml # Get list of namespaces and drill into one kubectl get ns kubectl describe ns uat
-
Assign CPU, Memory and Storage Quotas to Namespaces
# Create namespace quotas kubectl apply -f namespace-quotas.yaml # Get list of namespaces and drill into one kubectl get ns kubectl describe ns dev
-
Test out Limits and Quotas in dev Namespace
# Test Limits - Forbidden due to assignment of CPU too low kubectl run nginx-limittest --image=nginx --restart=Never --replicas=1 --port=80 --requests='cpu=100m,memory=256Mi' -n dev # Test Limits - Pass due to automatic assignment within limits via defaults kubectl run nginx-limittest --image=nginx --restart=Never --replicas=1 --port=80 -n dev # Check running pod and dev Namespace Allocations kubectl get po -n dev kubectl describe ns dev # Test Quotas - Forbidden due to memory quota exceeded kubectl run nginx-quotatest --image=nginx --restart=Never --replicas=1 --port=80 --requests='cpu=500m,memory=1Gi' -n dev # Test Quotas - Pass due to memory within quota kubectl run nginx-quotatest --image=nginx --restart=Never --replicas=1 --port=80 --requests='cpu=500m,memory=512Mi' -n dev # Check running pod and dev Namespace Allocations kubectl get po -n dev kubectl describe ns dev
-
Clean up limits, quotas, pods
kubectl delete -f namespace-limitranges.yaml kubectl delete -f namespace-quotas.yaml kubectl delete po nginx-limittest nginx-quotatest -n dev kubectl describe ns dev kubectl describe ns uat kubectl describe ns prod
- The limits and quotas of a namespace can be found via the kubectl describe ns <...> command. You will also be able to see current allocations.
- If pods are not deploying then check to make sure that CPU, Memory and Storage amounts are within the limits and do not exceed the overall quota of the namespace.
- Kubernetes Namespaces
- Default CPU Requests and Limits for a Namespace
- Configure Min and Max CPU Constraints for a Namespace
- Configure Memory and CPU Quotas for a Namespace