Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding multiple tolerations on vizier-pems #1939

Open
happicamper opened this issue Jun 5, 2024 · 0 comments
Open

Adding multiple tolerations on vizier-pems #1939

happicamper opened this issue Jun 5, 2024 · 0 comments

Comments

@happicamper
Copy link

Is your feature request related to a problem? Please describe.
Hi Pixie Team! I'm a bit new with Pixie and wanted to be able to add nodeSelector and tolerations on my vizier-pem pods.
I'm currently using Helm values.yaml installation via Terraform.
So far I've seen the New relic documentation about Pixie patch here in which suggests to use patches. This works perfectly on nodeSelector however when using tolerations it doesn't work. Furthermore, I can't seem to find a way to add multiple tolerations

pixie-chart:
  enabled: true
    patches:
      vizier-pem: '{"spec": {"template": {"spec": { "nodeSelector": {"pixie": "allowed" }}}}}'

Describe the solution you'd like
The ability to add multiple tolerations on vizier-pem pods and can be retrieved dynamically for example the snippet below is where I apply tolerations on other components of nri-bundle. I'm using a list of tolerations that differ depending on the environment being used.

kube-state-metrics:
  enabled: true
  tolerations:
    %{for tolerationKey in tolerationKeys}
    - key: ${tolerationKey}
      operator: Exists
      effect: NoSchedule
    %{endfor}

Describe alternatives you've considered
Since I'm using Terraform, I've tried using resource local_file to generate the patch file with dynamic toleration values

# vizier-pem patch via kubernetes_manifest
resource "local_file" "vizier_patch_file" {
  filename   = "${path.module}/pixie-patch.yaml "
  content    = <<-EOT
  spec:
    template:
      spec:
        tolerations:
          %{for tolerationKey in local.node_selectors}
          - key: ${tolerationKey}
            operator: Exists
            effect: NoSchedule
          %{endfor}
  EOT
}

And then follow it up with null_resource local-exec to execute kubectl patch.

# Pixie Vizier-pem toleration patch
resource "null_resource" "vizier_pem_patch" {
  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command     = <<EOT
    kubectl patch daemonset vizier-pem --patch-file ${path.module}/pixie-patch.yaml -n monitoring-newrelic
    EOT
  }
}

This solution works however, the resource local_exec creates noise as it re-create the vizier patch file every time it doesn't match the tolerations present in the environment.

Additional context
I've checked on #598 however, it didn't work on my end. Any help or insight is greatly appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant