Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All task pods are getting launched on single node #8346

Open
jkuma145 opened this issue Oct 24, 2024 · 2 comments
Open

All task pods are getting launched on single node #8346

jkuma145 opened this issue Oct 24, 2024 · 2 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@jkuma145
Copy link

jkuma145 commented Oct 24, 2024

Expected Behavior

For each pipelineruns, Pods should be run on multiple nodes. I'm running 5 pipelinerun parallelly which is running all pods on single nodes. I have 5-6 nodes in my cluster where tekton is hosted.

Actual Behavior

Steps to Reproduce the Problem

  1. setp eventlistener, setup TriggerTemplate and refer to pipeline which should run your task on multiple nodes.

Additional Info

  • Kubernetes version:

  • 1.28.6

    Output of kubectl version:

Client Version: v1.29.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.25.6
  • Tekton Pipeline version:

    Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

v0.61.1

I'm trying to tekton cluster in HA even, but most of pods are running on single node only.

#tektonpipelines

@jkuma145 jkuma145 added the kind/bug Categorizes issue or PR as related to a bug. label Oct 24, 2024
@jkuma145 jkuma145 changed the title All pods are getting launched on single node All task pods are getting launched on single node Oct 24, 2024
@AlanGreene
Copy link
Member

Can you share an example of a PipelineRun exhibiting this behaviour? Depending on what it's doing this may be expected.

See https://tekton.dev/docs/pipelines/affinityassistants/

coschedule workspaces - When a PersistentVolumeClaim is used as volume source for a Workspace in a PipelineRun, all TaskRun pods within the PipelineRun that share the Workspace will be scheduled to the same Node. (Note: Only one pvc-backed workspace can be mounted to each TaskRun in this mode.)

coschedule workspaces is the default behaviour, but other options are available including disabling the affinity assistant as described on that page.

If your PipelineRuns are not using a PVC as described, please share more details to help understand and reproduce the behaviour.

@jkuma145
Copy link
Author

Hello
I'm running 5 pipelinerun parallely for 5 applications. Pipelineruns -> Taskruns -> Pods which are executing on single node for all 5 pipelinerun. I can understand that single pipelinerun can run multiple task and multiple pods on single node as we are sharing the workspace using coherent. but unexpected behavior is all pipelinerun running all pods on single node only.

attached pod sample

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants