Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics Exporter Bundles Remote Tasks Together and Labels Them as "Anonymous" #8215

Open
OsamaKhan220 opened this issue Aug 20, 2024 · 2 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@OsamaKhan220
Copy link

Description:

When running a pipeline with a mix of ClusterTask and remote Task definitions, the Tekton controller correctly exports metrics for the ClusterTask. However, metrics for the remote tasks are bundled together and labeled as "anonymous" in Prometheus.

Steps to Reproduce:

  1. Define a pipeline with both ClusterTask and remote Task references.
  2. Execute the pipeline.
  3. Observe the metrics exported by the Tekton controller.

Expected Behavior:
Each task, whether ClusterTask or remote Task, should have its own distinct metrics exported, labeled with the appropriate task name (e.g., git-clone or stakater-create-git-tag).

Actual Behavior:
Metrics for remote tasks are bundled together and labeled as "anonymous," making it difficult to distinguish between them.

Example Pipeline Snippet:

Copy code
- name: fetch-repository
  taskRef:
    name: git-clone
    kind: ClusterTask
  params:
    - name: depth
      value: "0"
  
- name: create-git-tag
  taskRef:
    name: stakater-create-git-tag
    kind: Task
  
- name: create-environment
  taskRef:
    kind: Task
    name: stakater-create-environment
  params:

@vdemeester vdemeester added the kind/bug Categorizes issue or PR as related to a bug. label Aug 21, 2024
@chengjoey
Copy link
Member

+1, it seems that not only clusterTask or remoteTask, but also ordinary tasks will have anonymous

tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="900"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="1800"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="3600"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="5400"} 6
100 88827    0 88827    0     0  18.4M      0 --:--:-- --:--:-- --:--:-- 21.1M
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="10800"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="21600"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="43200"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="86400"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_bucket{namespace="default",pipeline="volume-from-template",status="success",task="anonymous",le="+Inf"} 6
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_sum{namespace="default",pipeline="volume-from-template",status="success",task="anonymous"} 57.044900693
tekton_pipelines_controller_pipelinerun_taskrun_duration_seconds_count{namespace="default",pipeline="volume-from-template",status="success",task="anonymous"} 6
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-reader-pod",task="anonymous"} 0
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-writer-pod",task="anonymous"} 1000
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-xmdls-reader-pod",task="anonymous"} 0
tekton_pipelines_controller_taskruns_pod_latency_milliseconds{namespace="default",pod="run-with-template-xmdls-writer-pod",task="anonymous"} 1000

@chengjoey
Copy link
Member

  1. cluster task use different key

    if tr.Spec.TaskRef != nil {
    if tr.Spec.TaskRef.Kind == "ClusterTask" {
    tr.ObjectMeta.Labels[pipeline.ClusterTaskLabelKey] = meta.Name
    } else {
    tr.ObjectMeta.Labels[pipeline.TaskLabelKey] = meta.Name
    }
    }
    }

  2. taskrun.Spec.TaskSpec != nil, it may be created by pipelinerun, use tekton.dev/pipelineTask

kubectl describe taskrun run-with-template-writer
Name:         run-with-template-writer
Namespace:    default
Labels:       app.kubernetes.io/managed-by=tekton-pipelines
              tekton.dev/memberOf=tasks
              tekton.dev/pipeline=volume-from-template
              tekton.dev/pipelineRun=run-with-template
              tekton.dev/pipelineTask=writer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

3 participants