kubernetes_manifest crashes on ArgoCD's ApplicationSet with matrix generator (git + cluster)

[ivankorn]

Terraform Version, Provider Version and Kubernetes Version

Terraform version:  v1.9.3
Kubernetes provider version: v2.30.0
Kubernetes version: 1.29.6-gke.1326000	

Affected Resource(s)

• kubernetes_manifest
• opc_storage_volume

Terraform Configuration Files

resource "kubernetes_manifest" "argocd_application_set" {
  manifest = provider::kubernetes::manifest_decode(file("argocd/application-set.yaml"))
}

ApplicationSet Manifest (works with kubectl create -f)

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-git
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://github.com/<MASKED>/<MASKED>_kustomize ## WARN: real value is masked for privacy reasons
              revision: HEAD
              directories:
                - path: overlays/*/*
          - clusters:
              selector:
                matchLabels:
                  argocd.argoproj.io/secret-type: cluster
                  cluster-type: <MASKED> ## WARN: real value is masked for privacy reasons
  template:
    metadata:
      name: "{{.path.basename}}-{{.name}}"
    spec:
      project: "{{.name}}"
      source:
        repoURL: https://github.com/<MASKED>/<MASKED>_kustomize ## WARN: real value is masked for privacy reasons
        targetRevision: HEAD
        path: "overlays/{{.name}}/{{.path.basename}}"
      destination:
        name: "{{.name}}"
        namespace: <MASKED> ## WARN: real value is masked for privacy reasons
      syncPolicy:
        automated:
          allowEmpty: false
          prune: true
          selfHeal: true
        retry:
          backoff:
            duration: 5s
            factor: 2
            maxDuration: 3m
          limit: 5
        syncOptions:
          - CreateNamespace=true

Debug Output

debug.log

Panic Output

crash.log

Steps to Reproduce

1. install argocd e.g via helm https://github.com/argoproj/argo-helm/tree/main/charts/argo-cd
2. make ApplicationSet manifest as specified above
3. define kubernetes_manifest resource as specified above
4. terraform init
5. terraform plan or terraform apply

Expected Behavior

No crash. Plan gets rendered or resource gets provisioned into Kubernetes.

Actual Behavior

Both terraform plan and tarraform apply crash with the panic/debug output attached above

Important Factoids

Non-matrix, f.e. Cluster Generator works fine. To the best of my knowledge the problem is reproducible only with ArgoCD's Application Set with Matrix Generator. Other resources (both non-argocd and argocd) get provisioned just fine.

References

• kubernetes_manifest crashes during plan of argocd applicationset with matrix generator CR #2503

Note: My issue is slightly different because I'm using the new provider::kubernetes::manifest_decode provider function to define manifest. But I get very similar (or the same) error.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[ivankorn]

If anyone is interested in temporary workaround for this issue:

# Workaround for https://github.com/hashicorp/terraform-provider-kubernetes/issues/2580
module "kubectl-argocd-application-set" {
  source  = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"
  version = "~> 3.4"

  project_id        = local.gcp_project_id
  cluster_name      = module.gke.name
  cluster_location  = module.gke.location
  module_depends_on = [module.gke.endpoint]

  kubectl_create_command  = "kubectl create -f argocd/application-set.yaml"
  kubectl_destroy_command = "kubectl delete -f argocd/application-set.yaml"
  skip_download           = true
}