Skip to content

Commit

Permalink
Update docs to allow webhooks to enabled by the helm controller
Browse files Browse the repository at this point in the history
  • Loading branch information
engedaam committed Aug 12, 2024
1 parent 713a03b commit abcc9f4
Showing 1 changed file with 52 additions and 8 deletions.
60 changes: 52 additions & 8 deletions website/content/en/preview/upgrading/v1-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,18 +44,59 @@ Please read through the entire procedure before beginning the upgrade. There are

2. Review for breaking changes between v0.33 and v0.37: If you are already running Karpenter v0.37.x, you can skip this step. If you are running an earlier Karpenter version, you need to review the [Upgrade Guide]({{<ref "upgrade-guide#upgrading-to-0320" >}}) for each minor release.

3. Set environment variables for your cluster:
3. Set environment variables for your cluster to upgrade to the latest patch version of the current Karpenter version you're running on:
```bash
export KARPENTER_NAMESPACE=kube-system
export KARPENTER_VERSION=1.0.0
export KARPENTER_IAM_ROLE_ARN="arn:${AWS_PARTITION}:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter"
export AWS_PARTITION="aws" # if you are not using standard partitions, you may need to configure to aws-cn / aws-us-gov
export CLUSTER_NAME="${USER}-karpenter-demo"
export AWS_REGION="us-west-2"
export AWS_ACCOUNT_ID="$(aws sts get-caller-identity --query Account --output text)"
```
4. Update your existing policy using the following:
4. Set environment variables for upgrading to the latest patch version:
```bash
export KARPENTER_VERSION=<latest patch version of your current v1beta1 minor version>
```
6. Apply the latest patch version of your current minor version's Custom Resource Definitions (CRDs):

```bash
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
--set webhook.enabled=true \
--set webhook.serviceName=karpenter \
--set webhook.serviceNamespace="${KARPENTER_NAMESPACE}" \
--set webhook.port=8443
```


7. Upgrade Karpenter to the latest patch version of your current minor version's. At the end of this step, conversion webhooks will run but will not convert any version.
```bash
# Service account annotation can be dropped when using pod identity
helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
--set settings.clusterName=${CLUSTER_NAME} \
--set settings.interruptionQueue=${CLUSTER_NAME} \
--set controller.resources.requests.cpu=1 \
--set controller.resources.requests.memory=1Gi \
--set controller.resources.limits.cpu=1 \
--set controller.resources.limits.memory=1Gi \
--set webhook.enabled=true \
--set webhook.port=8443 \
--wait
```
8. Set environment variables for first upgrading to v1.0.0
```bash
export KARPENTER_VERSION=1.0.0
```
9. Update your existing policy using the following to the v1.0.0 controller policy:
Notable Changes to the IAM Policy include `SSM:GetParameter` to `SSM:GetParametersByPath` and additional tag-scoping for the `eks:eks-cluster-name` tag for instances and instance profiles.
```bash
Expand All @@ -68,20 +109,21 @@ Please read through the entire procedure before beginning the upgrade. There are
--parameter-overrides "ClusterName=${CLUSTER_NAME}"
```
5. Apply the v1.0.0 Custom Resource Definitions (CRDs):
10. Apply the v1.0.0 Custom Resource Definitions (CRDs):
```bash
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
```bash
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version "${KARPENTER_VERSION}" --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
--set webhook.enabled=true \
--set webhook.serviceName=karpenter \
--set webhook.serviceNamespace="${KARPENTER_NAMESPACE}" \
--set webhook.port=8443
```
6. Upgrade Karpenter to the new version. At the end of this step, conversion webhooks run to convert the Karpenter CRDs to v1.
11. Upgrade Karpenter to the new version. At the end of this step, conversion webhooks run to convert the Karpenter CRDs to v1.
```bash
helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \ # Service account annotion can be droped when using pod identity
--set settings.clusterName=${CLUSTER_NAME} \
--set settings.interruptionQueue=${CLUSTER_NAME} \
--set controller.resources.requests.cpu=1 \
Expand All @@ -95,7 +137,7 @@ Please read through the entire procedure before beginning the upgrade. There are
Karpenter has deprecated and moved a number of Helm values as part of the v1 release. Ensure that you upgrade to the newer version of these helm values during your migration to v1. You can find detail for all the settings that were moved in the [v1 Upgrade Reference]({{<ref "#helm-values" >}}).
{{% /alert %}}
7. Once upgraded, you won't need to roll your nodes to be compatible with v1.1.0, except if you have multiple NodePools with different `kubelet`s that are referencing the same EC2NodeClass. Karpenter has moved the `kubelet` to the EC2NodeClass in v1. NodePools with different `kubelet` referencing the same EC2NodeClass will be compatible with v1.0.0, but will not be in v1.1.0.
12. Once upgraded, you won't need to roll your nodes to be compatible with v1.1.0, except if you have multiple NodePools with different `kubelet`s that are referencing the same EC2NodeClass. Karpenter has moved the `kubelet` to the EC2NodeClass in v1. NodePools with different `kubelet` referencing the same EC2NodeClass will be compatible with v1.0.0, but will not be in v1.1.0.

When you have completed the migration to `1.0.0` CRDs, Karpenter will be able to serve both the `v1beta1` versions and the `v1` versions of NodePools, NodeClaims, and EC2NodeClasses.
The results of upgrading these CRDs include the following:
Expand Down Expand Up @@ -211,6 +253,7 @@ Since both v1beta1 and v1 will be served, `kubectl` will default to returning th
```bash
export KARPENTER_NAMESPACE="kube-system"
export KARPENTER_VERSION="<rollback version of karpenter>"
export KARPENTER_IAM_ROLE_ARN="arn:${AWS_PARTITION}:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter"
export CLUSTER_NAME="<name of your cluster>"
export TEMPOUT="$(mktemp)"
```
Expand Down Expand Up @@ -249,6 +292,7 @@ helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-cr
4. Rollback the Karpenter Controller
```bash
# Service account annotation can be dropped when using pod identity
helm upgrade --install karpenter oci://public.ecr.aws/karpenter/karpenter --version ${KARPENTER_VERSION} --namespace "${KARPENTER_NAMESPACE}" --create-namespace \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
--set settings.clusterName=${CLUSTER_NAME} \
Expand Down

0 comments on commit abcc9f4

Please sign in to comment.