diff --git a/website/content/en/docs/concepts/nodeclasses.md b/website/content/en/docs/concepts/nodeclasses.md index 790aaf07770d..5a8a978da6a2 100644 --- a/website/content/en/docs/concepts/nodeclasses.md +++ b/website/content/en/docs/concepts/nodeclasses.md @@ -383,8 +383,7 @@ kubelet: By default, the number of pods on a node is limited by both the number of networking interfaces (ENIs) that may be attached to an instance type and the number of IP addresses that can be assigned to each ENI. See [IP addresses per network interface per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for a more detailed information on these instance types' limits. {{% alert title="Note" color="primary" %}} -By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global [setting](./settings.md#configmap) within the karpenter-global-settings configmap: `aws.reservedENIs`. In the common case, `aws.reservedENIs` should be set to `"1"` if using Custom Networking. -{{% /alert %}} +By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global environment variable RESERVED_ENIS, see [Settings]({{}}). In the common case, RESERVED_ENIS should be set to "1" if using Custom Networking. {{% /alert %}} {{% alert title="Windows Support Notice" color="warning" %}} It's currently not possible to specify custom networking with Windows nodes. diff --git a/website/content/en/docs/reference/settings.md b/website/content/en/docs/reference/settings.md index a84521d42b34..4eef61fb3952 100644 --- a/website/content/en/docs/reference/settings.md +++ b/website/content/en/docs/reference/settings.md @@ -46,9 +46,35 @@ Karpenter uses [feature gates](https://kubernetes.io/docs/reference/command-line | Feature | Default | Stage | Since | Until | |-------------------------|---------|--------|---------|---------| | Drift | false | Alpha | v0.21.x | v0.32.x | -| Drift | true | Beta | v0.33.x | | +| Drift | true | Beta | v0.33.x | v0.37.x | | SpotToSpotConsolidation | false | Alpha | v0.34.x | | +{{% alert title="Note" color="primary" %}} +In v1, drift has been promoted to stable and the feature gate removed. Users can continue to control drift by using disruption budgets by reason. +Example: +```yaml +apiVersion: karpenter.sh/v1 +kind: NodePool +metadata: + name: default +spec: +… + disruption: + budgets: + - nodes: 10% + # On Weekdays during business hours, don't do any deprovisioning regarding drift. + - nodes: "0" + schedule: "0 9 * * mon-fri" + duration: 8h + reasons: + - Drifted + # during non-business hours do drift for up to 10% of nodes + - nodes: "10%" + reasons: + - Drifted +``` +{{% /alert %}} + ### Batching Parameters The batching parameters control how Karpenter batches an incoming stream of pending pods. Reducing these values may trade off a slightly faster time from pending pod to node launch, in exchange for launching smaller nodes. Increasing the values can do the inverse. Karpenter provides reasonable defaults for these values, but if you have specific knowledge about your workloads you can tweak these parameters to match the expected rate of incoming pods. diff --git a/website/content/en/preview/concepts/nodeclasses.md b/website/content/en/preview/concepts/nodeclasses.md index 15f751bcb5f8..4394a1ed7444 100644 --- a/website/content/en/preview/concepts/nodeclasses.md +++ b/website/content/en/preview/concepts/nodeclasses.md @@ -383,8 +383,7 @@ kubelet: By default, the number of pods on a node is limited by both the number of networking interfaces (ENIs) that may be attached to an instance type and the number of IP addresses that can be assigned to each ENI. See [IP addresses per network interface per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for a more detailed information on these instance types' limits. {{% alert title="Note" color="primary" %}} -By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global [setting](./settings.md#configmap) within the karpenter-global-settings configmap: `aws.reservedENIs`. In the common case, `aws.reservedENIs` should be set to `"1"` if using Custom Networking. -{{% /alert %}} +By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global environment variable RESERVED_ENIS, see [Settings]({{}}). In the common case, RESERVED_ENIS should be set to "1" if using Custom Networking. {{% /alert %}} {{% alert title="Windows Support Notice" color="warning" %}} It's currently not possible to specify custom networking with Windows nodes. diff --git a/website/content/en/preview/reference/settings.md b/website/content/en/preview/reference/settings.md index ef2a8606cc10..f39eebf3b8d9 100644 --- a/website/content/en/preview/reference/settings.md +++ b/website/content/en/preview/reference/settings.md @@ -44,9 +44,35 @@ Karpenter uses [feature gates](https://kubernetes.io/docs/reference/command-line | Feature | Default | Stage | Since | Until | |-------------------------|---------|--------|---------|---------| | Drift | false | Alpha | v0.21.x | v0.32.x | -| Drift | true | Beta | v0.33.x | | +| Drift | true | Beta | v0.33.x | v0.37.x | | SpotToSpotConsolidation | false | Alpha | v0.34.x | | +{{% alert title="Note" color="primary" %}} +In v1, drift has been promoted to stable and the feature gate removed. Users can continue to control drift by using disruption budgets by reason. +Example: +```yaml +apiVersion: karpenter.sh/v1 +kind: NodePool +metadata: + name: default +spec: +… + disruption: + budgets: + - nodes: 10% + # On Weekdays during business hours, don't do any deprovisioning regarding drift. + - nodes: "0" + schedule: "0 9 * * mon-fri" + duration: 8h + reasons: + - Drifted + # during non-business hours do drift for up to 10% of nodes + - nodes: "10%" + reasons: + - Drifted +``` +{{% /alert %}} + ### Batching Parameters The batching parameters control how Karpenter batches an incoming stream of pending pods. Reducing these values may trade off a slightly faster time from pending pod to node launch, in exchange for launching smaller nodes. Increasing the values can do the inverse. Karpenter provides reasonable defaults for these values, but if you have specific knowledge about your workloads you can tweak these parameters to match the expected rate of incoming pods. diff --git a/website/content/en/v1.0/concepts/nodeclasses.md b/website/content/en/v1.0/concepts/nodeclasses.md index 790aaf07770d..4f0c2ee3408a 100644 --- a/website/content/en/v1.0/concepts/nodeclasses.md +++ b/website/content/en/v1.0/concepts/nodeclasses.md @@ -383,7 +383,7 @@ kubelet: By default, the number of pods on a node is limited by both the number of networking interfaces (ENIs) that may be attached to an instance type and the number of IP addresses that can be assigned to each ENI. See [IP addresses per network interface per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) for a more detailed information on these instance types' limits. {{% alert title="Note" color="primary" %}} -By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global [setting](./settings.md#configmap) within the karpenter-global-settings configmap: `aws.reservedENIs`. In the common case, `aws.reservedENIs` should be set to `"1"` if using Custom Networking. +By default, the VPC CNI allocates IPs for a node and pods from the same subnet. With [VPC CNI Custom Networking](https://aws.github.io/aws-eks-best-practices/networking/custom-networking), the pods will receive IP addresses from another subnet dedicated to pod IPs. This approach makes it easier to manage IP addresses and allows for separate Network Access Control Lists (NACLs) applied to your pods. VPC CNI Custom Networking reduces the pod density of a node since one of the ENI attachments will be used for the node and cannot share the allocated IPs on the interface to pods. Karpenter supports VPC CNI Custom Networking and similar CNI setups where the primary node interface is separated from the pods interfaces through a global environment variable `RESERVED_ENIS`, see [Settings]({{}}). In the common case, `RESERVED_ENIS` should be set to `"1"` if using Custom Networking. {{% /alert %}} {{% alert title="Windows Support Notice" color="warning" %}} diff --git a/website/content/en/v1.0/reference/settings.md b/website/content/en/v1.0/reference/settings.md index 1569cd313431..08d483001a73 100644 --- a/website/content/en/v1.0/reference/settings.md +++ b/website/content/en/v1.0/reference/settings.md @@ -46,9 +46,35 @@ Karpenter uses [feature gates](https://kubernetes.io/docs/reference/command-line | Feature | Default | Stage | Since | Until | |-------------------------|---------|-------|---------|---------| | Drift | false | Alpha | v0.21.x | v0.32.x | -| Drift | true | Beta | v0.33.x | | +| Drift | true | Beta | v0.33.x | v0.37.x | | SpotToSpotConsolidation | false | Alpha | v0.34.x | | +{{% alert title="Note" color="primary" %}} +In v1, drift has been promoted to stable and the feature gate removed. Users can continue to control drift by using disruption budgets by reason. +Example: +```yaml +apiVersion: karpenter.sh/v1 +kind: NodePool +metadata: + name: default +spec: +… + disruption: + budgets: + - nodes: 10% + # On Weekdays during business hours, don't do any deprovisioning regarding drift. + - nodes: "0" + schedule: "0 9 * * mon-fri" + duration: 8h + reasons: + - Drifted + # during non-business hours do drift for up to 10% of nodes + - nodes: "10%" + reasons: + - Drifted +``` +{{% /alert %}} + ### Batching Parameters The batching parameters control how Karpenter batches an incoming stream of pending pods. Reducing these values may trade off a slightly faster time from pending pod to node launch, in exchange for launching smaller nodes. Increasing the values can do the inverse. Karpenter provides reasonable defaults for these values, but if you have specific knowledge about your workloads you can tweak these parameters to match the expected rate of incoming pods.