fix: some fixes for karpenter deploy

[kvvit]

Fixes for karpenter deploy

First error:

Error: karpenter/private failed to create kubernetes rest client for update of resource: resource [karpenter.sh/v1/EC2NodeClass] isn't valid for cluster, check the APIVersion and Kind fields are valid

   with kubectl_manifest.ec2nodeclass_private[0],
   on main.tf line 71, in resource "kubectl_manifest" "ec2nodeclass_private":
   71: resource "kubectl_manifest" "ec2nodeclass_private" {

Solution: set apiVersion: karpenter.k8s.aws/v1 for resources kubectl_manifest: ec2nodeclass_private and ec2nodeclass_public (file terraform/modules/k8s-karpenter/main.tf).

Second error:

 Error: karpenter/default failed to run apply: error when creating "/tmp/879917200kubectl_manifest.yaml": NodePool.karpenter.sh "default" is invalid: [spec.disruption.consolidateAfter: Required value, spec.template.spec.nodeClassRef.group: Required value, spec.template.spec.nodeClassRef.kind: Required value, <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]

   with kubectl_manifest.nodepool["default"],
   on main.tf line 135, in resource "kubectl_manifest" "nodepool":
  135: resource "kubectl_manifest" "nodepool" {

Solution: Added fielsds group and kind for private and public NodePools (file terragrunt/ACCOUNT_ID/us-east-1/demo/env.yaml)

        spec:
          nodeClassRef:
            group: karpenter.k8s.aws
            kind: EC2NodeClass
            name: public

Third error:

 Error: karpenter/ci failed to run apply: error when creating "/tmp/969255726kubectl_manifest.yaml": NodePool.karpenter.sh "ci" is invalid: [spec.disruption.consolidateAfter: Required value, <nil>: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation]

   with kubectl_manifest.nodepool["ci"],
   on main.tf line 135, in resource "kubectl_manifest" "nodepool":
  135: resource "kubectl_manifest" "nodepool" {

Solution: Added field consolidateAfter: 1m for private and public NodePools (file terragrunt/ACCOUNT_ID/us-east-1/demo/env.yaml)

     disruption:
       consolidationPolicy: WhenEmptyOrUnderutilized
       consolidateAfter: 1m

Last one:

I removed the unnecessary data block data "aws_ecrpublic_authorization_token" "token" {} and the fields from the helm_release resource: repository_username = data.aws_ecrpublic_authorization_token.token.user_name and repository_password = data.aws_ecrpublic_authorization_token.token.password. I tested these changes, and everything works perfectly.

I also updated the locals block for Karpenter. I set the chart value to oci://public.ecr.aws/karpenter/karpenter and left the repository field empty.

    chart         = try(var.helm.chart_name, "oci://public.ecr.aws/karpenter/karpenter")
    repository    = try(var.helm.repository, "")

This configuration has been tested across different Helm releases using OCI repositories, and everything works flawlessly. I successfully tested the Karpenter deployment from a Docker container in our sandbox account, and it was deployed without any issues.

There was another issue related to the aws_ecrpublic_authorization_token, related to aws provider. Since the public ECR is located in the us-east-1 region, this data doesn't work in other regions. An additional AWS provider with an alias is required to handle this.

[mglotov]

Why did you change this? What was the problem here?

[kvvit]

This configuration worked for github-runners that also use oci helm repository. And it's working now for deploy karpenter in the some project.

[kvvit]

I've added information about this changes in the description of the PR

[mglotov]

I'm not sure it works. Let's test it together

[kvvit]

I tested this yesterday in the sandbox account, and in the in one of the projects. All works fine. More over, this data works only for us-east-1 region. For other regions we need to use additional aws provider with alias.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud