From e702e9aa853def2201c921c9be94d718def95f46 Mon Sep 17 00:00:00 2001 From: Amanuel Engeda Date: Sun, 11 Aug 2024 16:52:57 -0700 Subject: [PATCH] Update docs to allow webhooks to enabled by the helm controller --- .../en/preview/upgrading/v1-migration.md | 67 +++++++++++++++++-- 1 file changed, 61 insertions(+), 6 deletions(-) diff --git a/website/content/en/preview/upgrading/v1-migration.md b/website/content/en/preview/upgrading/v1-migration.md index ee485ed12c52..b2d0f1307c80 100644 --- a/website/content/en/preview/upgrading/v1-migration.md +++ b/website/content/en/preview/upgrading/v1-migration.md @@ -48,15 +48,20 @@ Please read through the entire procedure before beginning the upgrade. There are ```bash export KARPENTER_NAMESPACE=kube-system - export KARPENTER_VERSION=1.0.0 + export KARPENTER_IAM_ROLE_ARN="arn:${AWS_PARTITION}:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter" export AWS_PARTITION="aws" # if you are not using standard partitions, you may need to configure to aws-cn / aws-us-gov export CLUSTER_NAME="${USER}-karpenter-demo" export AWS_REGION="us-west-2" export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)" ``` -4. Update your existing policy using the following: - Notable Changes to the IAM Policy include `SSM:GetParameter` to `SSM:GetParametersByPath` and additional tag-scoping for the `eks:eks-cluster-name` tag for instances and instance profiles. +4. Set environment variables for first upgrading to v0.37.1 + + ```bash + export KARPENTER_VERSION=0.37.1 + ``` + +5. Update your existing policy using the following: ```bash TEMPOUT=$(mktemp) @@ -68,20 +73,69 @@ Please read through the entire procedure before beginning the upgrade. There are --parameter-overrides "ClusterName=${CLUSTER_NAME}" ``` -5. Apply the v1.0.0 Custom Resource Definitions (CRDs): + +6. Apply the v0.37.1 Custom Resource Definitions (CRDs): ```bash helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" --create-namespace \ + --set webhook.enabled=false \ + --set webhook.serviceName=karpenter \ + --set webhook.serviceNamespace="${KARPENTER_NAMESPACE}" \ + --set webhook.port=8443 + ``` + + +7. Upgrade Karpenter to the v0.37.1. At the end of this step, conversion webhooks will run but will not convert any version. + + ```bash + helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace "${KARPENTER_NAMESPACE}" --create-namespace \ + --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \ + --set settings.clusterName=${CLUSTER_NAME} \ + --set settings.interruptionQueue=${CLUSTER_NAME} \ + --set controller.resources.requests.cpu=1 \ + --set controller.resources.requests.memory=1Gi \ + --set controller.resources.limits.cpu=1 \ + --set controller.resources.limits.memory=1Gi \ + --set webhook.enabled=true \ + --set webhook.port=8443 \ + --wait + ``` + +8. Set environment variables for first upgrading to v1.0.0 + + ```bash + export KARPENTER_VERSION=1.0.0 + ``` + + +9. Update your existing policy using the following: + Notable Changes to the IAM Policy include `SSM:GetParameter` to `SSM:GetParametersByPath` and additional tag-scoping for the `eks:eks-cluster-name` tag for instances and instance profiles. + + ```bash + TEMPOUT=$(mktemp) + curl -fsSL https://raw.githubusercontent.com/aws/karpenter-provider-aws/v"${KARPENTER_VERSION}"/website/content/en/docs/getting-started/getting-started-with-karpenter/cloudformation.yaml > ${TEMPOUT} \ + && aws cloudformation deploy \ + --stack-name "Karpenter-${CLUSTER_NAME}" \ + --template-file "${TEMPOUT}" \ + --capabilities CAPABILITY_NAMED_IAM \ + --parameter-overrides "ClusterName=${CLUSTER_NAME}" + ``` + +10. Apply the v1.0.0 Custom Resource Definitions (CRDs): + + ```bash + helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" --create-namespace \ --set webhook.enabled=true \ --set webhook.serviceName=karpenter \ --set webhook.serviceNamespace="${KARPENTER_NAMESPACE}" \ --set webhook.port=8443 ``` -6. Upgrade Karpenter to the new version. At the end of this step, conversion webhooks run to convert the Karpenter CRDs to v1. +11. Upgrade Karpenter to the new version. At the end of this step, conversion webhooks run to convert the Karpenter CRDs to v1. ```bash helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace "${KARPENTER_NAMESPACE}" --create-namespace \ + --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \ --set settings.clusterName=${CLUSTER_NAME} \ --set settings.interruptionQueue=${CLUSTER_NAME} \ --set controller.resources.requests.cpu=1 \ @@ -95,7 +149,7 @@ Please read through the entire procedure before beginning the upgrade. There are Karpenter has deprecated and moved a number of Helm values as part of the v1 release. Ensure that you upgrade to the newer version of these helm values during your migration to v1. You can find detail for all the settings that were moved in the [v1 Upgrade Reference]({{}}). {{% /alert %}} -7. Once upgraded, you won't need to roll your nodes to be compatible with v1.1.0, except if you have multiple NodePools with different `kubelet`s that are referencing the same EC2NodeClass. Karpenter has moved the `kubelet` to the EC2NodeClass in v1. NodePools with different `kubelet` referencing the same EC2NodeClass will be compatible with v1.0.0, but will not be in v1.1.0. +12. Once upgraded, you won't need to roll your nodes to be compatible with v1.1.0, except if you have multiple NodePools with different `kubelet`s that are referencing the same EC2NodeClass. Karpenter has moved the `kubelet` to the EC2NodeClass in v1. NodePools with different `kubelet` referencing the same EC2NodeClass will be compatible with v1.0.0, but will not be in v1.1.0. When you have completed the migration to `1.0.0` CRDs, Karpenter will be able to serve both the `v1beta1` versions and the `v1` versions of NodePools, NodeClaims, and EC2NodeClasses. The results of upgrading these CRDs include the following: @@ -211,6 +265,7 @@ Since both v1beta1 and v1 will be served, `kubectl` will default to returning th ```bash export KARPENTER_NAMESPACE="kube-system" export KARPENTER_VERSION="" +export KARPENTER_IAM_ROLE_ARN="arn:${AWS_PARTITION}:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter" export CLUSTER_NAME="" export TEMPOUT="$(mktemp)" ```