Skip to content

Commit

Permalink
docs: AL2023 AMI Family docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jmdeal committed Feb 22, 2024
1 parent 3a8dd3a commit 30ed09a
Showing 1 changed file with 131 additions and 1 deletion.
132 changes: 131 additions & 1 deletion website/content/en/preview/concepts/nodeclasses.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ Refer to the [NodePool docs]({{<ref "./nodepools" >}}) for settings applicable t

## spec.amiFamily

AMIFamily is a required field, dictating both the default bootstrapping logic for nodes provisioned through this `EC2NodeClass` but also selecting a group of recommended, latest AMIs by default. Currently, Karpenter supports `amiFamily` values `AL2`, `Bottlerocket`, `Ubuntu`, `Windows2019`, `Windows2022` and `Custom`. GPUs are only supported by default with `AL2` and `Bottlerocket`. The `AL2` amiFamily does not support ARM64 GPU instance types unless you specify custom [`amiSelectorTerms`]({{<ref "#specamiselectorterms" >}}). Default bootstrapping logic is shown below for each of the supported families.
AMIFamily is a required field, dictating both the default bootstrapping logic for nodes provisioned through this `EC2NodeClass` but also selecting a group of recommended, latest AMIs by default. Currently, Karpenter supports `amiFamily` values `AL2`, `AL2023`, `Bottlerocket`, `Ubuntu`, `Windows2019`, `Windows2022` and `Custom`. GPUs are only supported by default with `AL2` and `Bottlerocket`. The `AL2` amiFamily does not support ARM64 GPU instance types unless you specify custom [`amiSelectorTerms`]({{<ref "#specamiselectorterms" >}}). Default bootstrapping logic is shown below for each of the supported families.

### AL2

Expand All @@ -184,6 +184,34 @@ exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
--//--
```

### AL2023

```yaml
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="//"
--//
Content-Type: application/node.eks.aws
# Karpenter Generated NodeConfig
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
cluster:
name: test-cluster
apiServerEndpoint: https://example.com
certificateAuthority: ca-bundle
cidr: 10.100.0.0/16
kubelet:
config:
maxPods: 110
flags:
- --node-labels=karpenter.sh/capacity-type=on-demand,karpenter.sh/nodepool=test
--//--
```

### Bottlerocket

```toml
Expand Down Expand Up @@ -581,6 +609,17 @@ spec:
encrypted: true
```

### AL2023
```yaml
spec:
blockDeviceMappings:
- deviceName: /dev/xvda
ebs:
volumeSize: 20Gi
volumeType: gp3
encrypted: true
```

### Bottlerocket
```yaml
spec:
Expand Down Expand Up @@ -646,6 +685,9 @@ The disks must be formatted & mounted in a RAID0 and be the underlying filesyste

On AL2, Karpenter automatically configures the disks through an additional boostrap argument (`--local-disks raid0`). The device name is `/dev/md/0` and its mount point is `/mnt/k8s-disks/0`. You should ensure any additional disk setup does not interfere with these.

#### AL2023
On AL2023, local disks are configured as raid0 by default, regardless of this field's value. Like AL2, the device name is `dev/md/0` and its mount point is `/mnt/k8s-disks/0`. You should ensure any additional disk setup does not interfere with these.

#### Others

For all other AMI families, you must configure the disks yourself. Check out the [`setup-local-disks`](https://github.com/awslabs/amazon-eks-ami/blob/master/files/bin/setup-local-disks) script in [amazon-eks-ami](https://github.com/awslabs/amazon-eks-ami) to see how this is done for AL2.
Expand Down Expand Up @@ -775,6 +817,94 @@ spec:
```
{{% /alert %}}

### AL2023

* Your UserData may be in one of three formats: a [MIME multi part archive](https://cloudinit.readthedocs.io/en/latest/topics/format.html#mime-multi-part-archive), a NodeConfig YAML / JSON string, or a shell script.
* Karpenter will transform your custom UserData as a MIME part, if necessary, and then merge a generated MIME part to the **beginning** of the the generated MIME multi part archive. Since `nodeadm`, the bootstrap for the AL2023 EKS optimized AMI, merges NodeConfig objects from top to bottom any fields specified in the custom UserData will override Karpenter generated fields. Take caution when overriding these fields, misconfiguration can result in Nodes being unable to connect to your cluster.

#### Passed-in UserData (NodeConfig)

```yaml
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
kubelet:
config:
maxPods: 42
```

#### Passed-in UserData (Bash)

```shell
#!/bin/bash
echo "Hello, AL2023!"
```

#### Passed-in UserData (MIME)

```shell
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="//"
--//
Content-Type: application/node.eks.aws
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
kubelet:
config:
maxPods: 42
--//
Content-Type: text/x-shellscript; charset="us-ascii"
#!/bin/bash
echo "Hello, AL2023!"
--//
```

#### Merged UserData

```shell
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="//"
--//
Content-Type: application/node.eks.aws
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
cluster:
apiServerEndpoint: https://test-cluster
certificateAuthority: cluster-ca
cidr: 10.100.0.0/16
name: test-cluster
kubelet:
config:
clusterDNS:
- 10.100.0.10
maxPods: 118
flags:
- --node-labels="karpenter.sh/capacity-type=on-demand,karpenter.sh/nodepool=default"
--//
Content-Type: application/node.eks.aws
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
kubelet:
config:
maxPods: 42
--//
Content-Type: text/x-shellscript; charset="us-ascii"
#!/bin/bash
echo "Hello, AL2023!"
--//--
```

### Bottlerocket

* Your UserData must be valid TOML.
Expand Down

0 comments on commit 30ed09a

Please sign in to comment.