Skip to content

Commit

Permalink
docs: add missing resource tag associated with spot (#5402)
Browse files Browse the repository at this point in the history
  • Loading branch information
@useradd-temp
useradd-temp authored Jan 1, 2024
1 parent 5b81ee3 commit 66a1518
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 23 deletions.
13 changes: 7 additions & 6 deletions website/content/en/docs/reference/cloudformation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ Someone wanting to add Karpenter to an existing cluster, instead of using `cloud

The AllowScopedEC2InstanceActions statement ID (Sid) identifies a set of EC2 resources that are allowed to be accessed with
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) and [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html) actions.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `spot-instances-request`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.

```json
{
Expand All @@ -122,7 +122,6 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read
"Resource": [
"arn:${AWS::Partition}:ec2:${AWS::Region}::image/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}::snapshot/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:security-group/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:subnet/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
Expand All @@ -138,7 +137,7 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read

The AllowScopedEC2InstanceActionsWithTags Sid allows the
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html), [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html), and [CreateLaunchTemplate](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateLaunchTemplate.html)
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, or `launch-template` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, `launch-template` or `spot-instances-request` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.

```json
{
Expand All @@ -149,7 +148,8 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": [
"ec2:RunInstances",
Expand All @@ -170,7 +170,7 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
#### AllowScopedResourceCreationTagging

The AllowScopedResourceCreationTagging Sid allows EC2 [CreateTags](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateTags.html)
actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-template` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.
actions on `fleet`, `instance`, `volume`, `network-interface`, `launch-template` and `spot-instances-request` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.

```json
{
Expand All @@ -181,7 +181,8 @@ actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-templ
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": "ec2:CreateTags",
"Condition": {
Expand Down
23 changes: 12 additions & 11 deletions website/content/en/preview/reference/cloudformation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ That name would then be appended to any name below where `${ClusterName}` is inc

* Partition: Any time an ARN is used, it includes the [partition name](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/partitions.html) to identify where the object is found. In most cases, that partition name is `aws`. However, it could also be `aws-cn` (for China Regions) or `aws-us-gov` (for AWS GovCloud US Regions).

## Node Authorization
## Node Authorization

The following sections of the `cloudformation.yaml` file set up IAM permissions for Kubernetes nodes created by Karpenter.
In particular, this involves setting up a node role that can be attached and passed to instance profiles that Karpenter generates at runtime:
Expand Down Expand Up @@ -79,7 +79,7 @@ The role created here includes several AWS managed policies, which are designed
If you were to use a node role from an existing cluster, you could skip this provisioning step and pass this node role to any EC2NodeClasses that you create. Additionally, you would ensure that the [Controller Policy]({{< relref "#controllerpolicy" >}}) has `iam:PassRole` permission to the role attached to the generated instance profiles.

## Controller Authorization
## Controller Authorization

This section sets the AWS permissions for the Karpenter Controller. When used in the Getting Started guide, `eksctl` uses these permissions to create a service account (karpenter) that is combined with the KarpenterControllerPolicy.

Expand Down Expand Up @@ -113,7 +113,7 @@ Someone wanting to add Karpenter to an existing cluster, instead of using `cloud

The AllowScopedEC2InstanceActions statement ID (Sid) identifies a set of EC2 resources that are allowed to be accessed with
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) and [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html) actions.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `spot-instances-request`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.

```json
{
Expand All @@ -122,7 +122,6 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read
"Resource": [
"arn:${AWS::Partition}:ec2:${AWS::Region}::image/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}::snapshot/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:security-group/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:subnet/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
Expand All @@ -136,9 +135,9 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read

#### AllowScopedEC2InstanceActionsWithTags

The AllowScopedEC2InstanceActionsWithTags Sid allows the
The AllowScopedEC2InstanceActionsWithTags Sid allows the
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html), [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html), and [CreateLaunchTemplate](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateLaunchTemplate.html)
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, or `launch-template` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, `launch-template` or `spot-instances-request` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.

```json
{
Expand All @@ -149,7 +148,8 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": [
"ec2:RunInstances",
Expand All @@ -170,7 +170,7 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
#### AllowScopedResourceCreationTagging

The AllowScopedResourceCreationTagging Sid allows EC2 [CreateTags](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateTags.html)
actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-template` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.
actions on `fleet`, `instance`, `volume`, `network-interface`, `launch-template` and `spot-instances-request` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.

```json
{
Expand All @@ -181,7 +181,8 @@ actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-templ
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": "ec2:CreateTags",
"Condition": {
Expand Down Expand Up @@ -404,7 +405,7 @@ Also, `karpenter.k8s.aws/ec2nodeclass` must be set to some value. This ensures t

#### AllowScopedInstanceProfileActions

The AllowScopedInstanceProfileActions Sid gives the Karpenter controller permission to perform [`iam:AddRoleToInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_AddRoleToInstanceProfile.html), [`iam:RemoveRoleFromInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_RemoveRoleFromInstanceProfile.html), and [`iam:DeleteInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_DeleteInstanceProfile.html) actions,
The AllowScopedInstanceProfileActions Sid gives the Karpenter controller permission to perform [`iam:AddRoleToInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_AddRoleToInstanceProfile.html), [`iam:RemoveRoleFromInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_RemoveRoleFromInstanceProfile.html), and [`iam:DeleteInstanceProfile`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_DeleteInstanceProfile.html) actions,
provided that the request is made to a cluster with `kubernetes.io/cluster/${ClusterName` set to owned and is made in the current region.
Also, `karpenter.k8s.aws/ec2nodeclass` must be set to some value. This permission is further enforced by the `iam:PassRole` permission. If Karpenter attempts to add a role to an instance profile that it doesn't have `iam:PassRole` permission on, that call will fail. Therefore, if you configure Karpenter to use a new role through the `EC2NodeClass`, ensure that you also specify that role within your `iam:PassRole` permission.

Expand Down Expand Up @@ -459,7 +460,7 @@ The AllowAPIServerEndpointDiscovery Sid allows the Karpenter controller to get t
}
```

## Interruption Handling
## Interruption Handling

Settings in this section allow the Karpenter controller to stand-up an interruption queue to receive notification messages from other AWS services about the health and status of instances. For example, this interruption queue allows Karpenter to be aware of spot instance interruptions that are sent 2 minutes before spot instances are reclaimed by EC2. Adding this queue allows Karpenter to be proactive in migrating workloads to new nodes.
See the [Interruption]({{< relref "../concepts/disruption#interruption" >}}) section of the Disruption page for details.
Expand Down
13 changes: 7 additions & 6 deletions website/content/en/v0.33/reference/cloudformation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ Someone wanting to add Karpenter to an existing cluster, instead of using `cloud

The AllowScopedEC2InstanceActions statement ID (Sid) identifies a set of EC2 resources that are allowed to be accessed with
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) and [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html) actions.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `spot-instances-request`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.
For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read (but not create) `image`, `snapshot`, `security-group`, `subnet` and `launch-template` EC2 resources, scoped for the particular AWS partition and region.

```json
{
Expand All @@ -122,7 +122,6 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read
"Resource": [
"arn:${AWS::Partition}:ec2:${AWS::Region}::image/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}::snapshot/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:security-group/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:subnet/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
Expand All @@ -138,7 +137,7 @@ For `RunInstances` and `CreateFleet` actions, the Karpenter controller can read

The AllowScopedEC2InstanceActionsWithTags Sid allows the
[RunInstances](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html), [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html), and [CreateLaunchTemplate](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateLaunchTemplate.html)
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, or `launch-template` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.
actions requested by the Karpenter controller to create all `fleet`, `instance`, `volume`, `network-interface`, `launch-template` or `spot-instances-request` EC2 resources (for the partition and region), and requires that the `kubernetes.io/cluster/${ClusterName}` tag be set to `owned` and a `karpenter.sh/nodepool` tag be set to any value. This ensures that Karpenter is only allowed to create instances for a single EKS cluster.

```json
{
Expand All @@ -149,7 +148,8 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": [
"ec2:RunInstances",
Expand All @@ -170,7 +170,7 @@ actions requested by the Karpenter controller to create all `fleet`, `instance`,
#### AllowScopedResourceCreationTagging

The AllowScopedResourceCreationTagging Sid allows EC2 [CreateTags](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateTags.html)
actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-template` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.
actions on `fleet`, `instance`, `volume`, `network-interface`, `launch-template` and `spot-instances-request` resources, While making `RunInstance`, `CreateFleet`, or `CreateLaunchTemplate` calls. Additionally, this ensures that resources can't be tagged arbitrarily by Karpenter after they are created.

```json
{
Expand All @@ -181,7 +181,8 @@ actions on `fleet`, `instance`, `volume`, `network-interface`, and `launch-templ
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:instance/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:volume/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:network-interface/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*"
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:launch-template/*",
"arn:${AWS::Partition}:ec2:${AWS::Region}:*:spot-instances-request/*"
],
"Action": "ec2:CreateTags",
"Condition": {
Expand Down

0 comments on commit 66a1518

Please sign in to comment.