Skip to content

Commit

Permalink
More review comments resolved
Browse files Browse the repository at this point in the history
  • Loading branch information
@chrisnegus
chrisnegus committed Oct 18, 2023
1 parent ae605f7 commit bcd5f5e
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 50 deletions.
25 changes: 1 addition & 24 deletions website/content/en/preview/upgrade/compatibility.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: >
# Compatibility

To make upgrading easier we aim to minimize the introduction of breaking changes.
Before you begin upgrading Karpenter, consider Karpenter compatibility issues related to Kubernetes, the NodePool API (previously Provisioner), and Kubernetes Custom Resource Definitions (CRDs) applied through Helm Charts.
Before you begin upgrading Karpenter, consider Karpenter compatibility issues related to Kubernetes and the NodePool API (previously Provisioner).

## Compatibility Matrix

Expand Down Expand Up @@ -42,29 +42,6 @@ minor version when we release a new feature.

Users should therefore check to see if there is a breaking change every time they are upgrading to a new minor version.

### Custom Resource Definition (CRD) Upgrades

Karpenter ships with a few Custom Resource Definitions (CRDs). These CRDs are published:
* As an independent helm chart [karpenter-crd](https://gallery.ecr.aws/karpenter/karpenter-crd) - [source](https://github.com/aws/karpenter/blob/main/charts/karpenter-crd) that can be used by Helm to manage the lifecycle of these CRDs.
* To upgrade or install `karpenter-crd` run:
```
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version vx.y.z --namespace karpenter --create-namespace
```

{{% alert title="Note" color="warning" %}}
< If you get the error `invalid ownership metadata; label validation error:` while installing the `karpenter-crd` chart from an older version of Karpenter, follow the [Troubleshooting Guide]({{<ref "../troubleshooting#helm-error-when-installing-the-karpenter-crd-chart" >}}) for details on how to resolve these errors.
{{% /alert %}}

* As part of the helm chart [karpenter](https://gallery.ecr.aws/karpenter/karpenter) - [source](https://github.com/aws/karpenter/blob/main/charts/karpenter/crds). Helm [does not manage the lifecycle of CRDs using this method](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/), the tool will only install the CRD during the first installation of the helm chart. Subsequent chart upgrades will not add or remove CRDs, even if the CRDs have changed. When CRDs are changed, we will make a note in the version's upgrade guide.

In general, you can reapply the CRDs in the `crds` directory of the Karpenter helm chart:

```shell
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.sh_nodepools.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.sh_nodeclaims.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
```

### How Do We Break Incompatibility?

When there is a breaking change we will:
Expand Down
16 changes: 13 additions & 3 deletions website/content/en/preview/upgrade/upgrade-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,24 @@ Add `~/go/bin` to your $PATH, if you have not already done so.
aws iam attach-role-policy --role-name "${ROLE_NAME}" --policy-arn "${POLICY_ARN}"
```

5. Apply the v0.32.0 CRDs in the crds directory of the Karpenter helm chart:
5. Apply the v0.32.0 Custom Resource Definitions (CRDs) in the crds directory of the Karpenter helm chart. Here are the ways you can do this:

```bash
* As an independent helm chart [karpenter-crd](https://gallery.ecr.aws/karpenter/karpenter-crd) - [source](https://github.com/aws/karpenter/blob/main/charts/karpenter-crd) that can be used by Helm to manage the lifecycle of these CRDs. To upgrade or install `karpenter-crd` run:
```
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version vx.y.z --namespace karpenter --create-namespace
```

{{% alert title="Note" color="warning" %}}
< If you get the error `invalid ownership metadata; label validation error:` while installing the `karpenter-crd` chart from an older version of Karpenter, follow the [Troubleshooting Guide]({{<ref "../troubleshooting#helm-error-when-installing-the-karpenter-crd-chart" >}}) for details on how to resolve these errors.
{{% /alert %}}

* As part of the helm chart [karpenter](https://gallery.ecr.aws/karpenter/karpenter) - [source](https://github.com/aws/karpenter/blob/main/charts/karpenter/crds). Helm [does not manage the lifecycle of CRDs using this method](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/), the tool will only install the CRD during the first installation of the helm chart. Subsequent chart upgrades will not add or remove CRDs, even if the CRDs have changed. When CRDs are changed, we will make a note in the version's upgrade guide. In general, ou can reapply the CRDs in the `crds` directory of the Karpenter helm chart:
```bash
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.sh_nodepools.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.sh_nodeclaims.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter{{< githubRelRef >}}pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
```

6. Upgrade Karpenter to the new version:
```bash
Expand Down
38 changes: 15 additions & 23 deletions website/content/en/preview/upgrade/v1beta1-reference.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,12 @@ For a more general understanding of Karpenter's compatibility, see the [Compatib

Use the information below to help migrate your Karpenter v1alpha assets to v1beta1.


### Custom Resource Definition (CRD) Upgrades

Karpenter ships with a few Custom Resource Definitions (CRDs). Starting with v0.32.0, CRDs representing Provisioners, Machines, and AWS Node Templates are replaced by those for Node Pools, Node Claims, and EC2 Node classes, respectively.
You can find these CRDs on [Karpenter APIs CRD](https://github.com/aws/karpenter/tree/main/pkg/apis/crds) page.

The [Karpenter Upgrade Guide}(]({{< relref "upgrade-guide" >}}) describes how to install the new CRDs using `kubectl`. However, you can instead install those CRDs as an independent helm chart [karpenter-crd](https://gallery.ecr.aws/karpenter/karpenter-crd), that can be used by Helm to manage the lifecycle of these CRDs. See [source](https://github.com/aws/karpenter/blob/main/charts/karpenter-crd) for the source code,
Karpenter ships with a few Custom Resource Definitions (CRDs). Starting with v0.32.0, CRDs representing Provisioners, Machines, and AWS Node Templates are replaced by those for NodePools, NodeClaims, and EC2Nodeclasses, respectively.
You can find these CRDs by visiting the [Karpenter GitHub repository](https://github.com/aws/karpenter/tree/main/pkg/apis/crds).

To upgrade or install `karpenter-crd` run:

```
helm upgrade --install karpenter-crd oci://public.ecr.aws/karpenter/karpenter-crd --version vx.y.z --namespace karpenter --create-namespace
```

{{% alert title="Note" color="warning" %}}
< If you get the error `invalid ownership metadata; label validation error:` while installing the `karpenter-crd` chart from an older version of Karpenter, follow the [Troubleshooting Guide]({{<ref "../troubleshooting#helm-error-when-installing-the-karpenter-crd-chart" >}}) for details on how to resolve these errors.
{{% /alert %}}
The [Karpenter Upgrade Guide}(]({{< relref "upgrade-guide" >}}) describes how to install the new CRDs.

## Annotations, Labels, and Status Conditions

Expand Down Expand Up @@ -168,7 +157,6 @@ spec:

The Karpenter `spec.provider` field has been deprecated since version v0.7.0 and is now removed in the new `NodePool` resource. Any of the fields that you could specify within the `spec.provider` field are now available in the separate `NodeClass` resource.


**Provider example (v1alpha)**

```
Expand Down Expand Up @@ -217,7 +205,6 @@ This field works the same as the `ttlSecondsAfterEmpty` field except this field
* `Never`: This disables empty consolidation.
* Duration String (e.g. “10m”, “1s”): This enables empty consolidation for the time specified.


**ttlSecondsAfterEmpty example (v1alpha)**

```
Expand Down Expand Up @@ -310,10 +297,9 @@ Karpenter now statically defaults some fields in the v1beta1 if they are not spe

### spec.template.spec.requirements Defaults Dropped

Karpenter v1beta1 drops the defaulting logic for the node requirements that were shipped by default with Provisioners. Previously, Karpenter would create dynamic defaulting in the following cases. If multiple of these cases were satisfied, those default requirements would be combined:

Karpenter v1beta1 drops the defaulting logic for the node requirements that were shipped by default with Provisioners in v1alpha5. Previously, Karpenter would create dynamic defaulting in the following cases. If multiple of these cases were satisfied, those default requirements would be combined:

* If you don’t specify any instance type requirement:
* If you didn't specify any instance type requirement:

```
spec:
Expand All @@ -326,7 +312,7 @@ Karpenter v1beta1 drops the defaulting logic for the node requirements that were
values: ["2"]
```

* If you don’t specify any capacity type requirement (`karpenter.sh/capacity-type`):
* If you didn’t specify any capacity type requirement (`karpenter.sh/capacity-type`):

```
spec:
Expand All @@ -336,7 +322,7 @@ Karpenter v1beta1 drops the defaulting logic for the node requirements that were
values: ["on-demand"]
```

* If you don’t specify any OS requirement (`kubernetes.io/os`):
* If you didn’t specify any OS requirement (`kubernetes.io/os`):
```
spec:
requirements:
Expand All @@ -345,7 +331,7 @@ Karpenter v1beta1 drops the defaulting logic for the node requirements that were
values: ["linux"]
```

* If you don’t specify any architecture requirement (`kubernetes.io/arch`):
* If you didn’t specify any architecture requirement (`kubernetes.io/arch`):
```
spec:
requirements:
Expand Down Expand Up @@ -373,7 +359,13 @@ Note that:

The Karpenter `spec.instanceProfile` field has been removed from the EC2NodeClass in favor of the `spec.role` field. Karpenter is also removing support for the `defaultInstanceProfile` specified globally in the karpenter-global-settings, making the `spec.role` field required for all EC2NodeClasses.

Karpenter will now auto-generate the instance profile in your EC2NodeClass, given the role that you specify.
Karpenter will now auto-generate the instance profile in your EC2NodeClass, given the role that you specify. To find the role, type:

```
export INSTANCE_PROFILE_NAME=KarpenterNodeInstanceProfile-bob-karpenter-demo
aws iam get-instance-profile --instance-profile-name $INSTANCE_PROFILE_NAME --query "InstanceProfile.Roles[0].RoleName"
KarpenterNodeRole-bob-karpenter-demo
```

**instanceProfile example (v1alpha)**

Expand Down

0 comments on commit bcd5f5e

Please sign in to comment.