-
Notifications
You must be signed in to change notification settings - Fork 579
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
catlin-validate task to lint task yaml's or set of task yamls #1178
Open
pratiktest
wants to merge
2
commits into
tektoncd:main
Choose a base branch
from
pratiktest:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| # Catlin Validate | ||
|
|
||
| * [Catlin](https://github.com/tektoncd/catlin) is a command-line tool that Lints Tekton Resources and Catalogs. | ||
| * Catlin Validate task supports running catlin validate for | ||
| * An isolated task yaml that you want to submit to tekton catalog | ||
| * A set of tasks that you want to submit to tekton catalog | ||
|
|
||
| ## Catlin validate use cases | ||
|
|
||
| ### Run catlin validate to quickly validate a sample task | ||
| * Imagine you are developing a task and you quickly want to run a catlin validate to check if this task can be submitted to the tekton catalog | ||
| * Without downloading catlin or having any knowledge on how to run catlin you can check if your task passes all catlin checks by simply running a taskrun | ||
| * A working taskrun is provided in the [tests](../0.1/tests/taskrun-success.yaml) folder. | ||
| * [This](../0.1/tests/taskrun-success.yaml) taskrun validates a sample hello task. | ||
| * The hello task yaml which we want to validate is injected in the source workspace in the appropriate folder structure using [this](../0.1/tests/source-success-task-configmap.yaml) configMap. | ||
| * The filepath of the hello.yaml is supplied as a catlin-input using [this](../0.1/tests/input-configmap.yaml) configMap | ||
| * Thus without knowing anything about how to run catlin validate you can quickly lint your task yaml by simply following above steps | ||
|
|
||
| ### Integrate catlin as a part of CI/CD pipelines for task | ||
| * You may have an internal tekton catalog or want to contribute to open source tekton catalog and before submitting your task to these you might want to run catlin validate checks as a part of a PR pipeline | ||
| * To acheive this you can easily integrate catlin-validate task to validate your task yamls as part of a PR CI/CD tekton pipeline. | ||
|
|
||
|
|
||
| ### Run catlin validate for multiple task submissions in your tekton catalog | ||
| * You might have multiple tasks submitted as a part of a PR to the tekton catalog | ||
| * Catlin Validate Task can be run to validate all the task files in the inputFile provided to the task | ||
| * Simply put the paths of all tasks in $(params.inputFile) and catlin validate task will lint all the tasks | ||
| * Check [Results](#results) to see what is the final result of the task when multiple tasks are provided as input to catlin-validate | ||
|
|
||
|
|
||
|
|
||
| ## Parameters | ||
|
|
||
| ### outputFile | ||
| Name of the file that holds catlin validate output for each task. This file will be available in [catlin-output](#catlin-output) workspace | ||
| ### inputFile | ||
| File that contains catlin input. This file contains relative paths to the task yaml's delimited by newline. Paths should adhere to the tekton task catalog standards. Check [input-configmap](../0.1/tests/input-configmap.yaml) for an example of an inputFile | ||
|
|
||
| ### ignoreWarnings | ||
| Ignores any warnings in catlin output. If catlin output has only success and warnings, all warnings are ignored and final catlin output will be success | ||
|
|
||
|
|
||
| ## Workspaces | ||
|
|
||
| ### source | ||
| This workspace contains files on which catlin will be run. This can be the git repo of your tekton catalog. task yamls in this folder should have a path that follows tekton-catalog folder structure. Check [this](../0.1/tests/source-success-task-configmap.yaml) configmap and how it is injected into the cource workspace in [this](../0.1/tests/taskrun-success.yaml) taskrun as an example | ||
| ### catlin-input | ||
| A workspace that contains [$(params.inputFile)](#inputfile) Check [this](../0.1/tests/input-configmap.yaml) configmap and how it is injected onto the catlin-input workspace in [this](../0.1/tests/taskrun-success.yaml) taskrun as an example | ||
| ### catlin-output | ||
| A workspace that contains output of catlin validate. Output of each catlin validate command is sent to [$(params.outputFle)](#outputfile) in this workspace | ||
|
|
||
| ## Results | ||
|
|
||
| ### catlin-status | ||
| * catlin-status contains the final catlin-validate task result Success, Warning or Failure which will depend on the cumulative output of catlin validate on all tasks | ||
| * Final result will be success if there is no error and no warning for any of the task specified in [inputFile](#inputfile). | ||
| * Final result will be warning if there is no error and a warning for any of the task specified in [inputFile](#inputfile). | ||
| * Final result will be failure if there an error in even one of the tasks specified in [inputFile](#inputfile) | ||
| * Output for each catlin validate commmand is stored in [outFile](#outputfile) in [catlin-output](#catlin-output) workspace | ||
|
|
||
|
|
||
| ## Platforms | ||
|
|
||
| The Task can be run on `linux/amd64` platform. | ||
|
|
||
| ## Example | ||
| * [tests](../0.1/tests) folder contains an example which you can use as a reference to create a taskrun that will lint your task before submitting it to a tekton catalog | ||
| * You will need a kubernetes cluster to test the catlin task. You can use [minikube](https://minikube.sigs.k8s.io/docs/start/) or [Kind](https://kind.sigs.k8s.io/) to install a kubernetes cluster locally. | ||
|
|
||
| ### Catlin Validate Success | ||
| * Follow the below steps to run the sample taskrun in your local cluster which validates a properly constructed task that adheres to catlin linting standards | ||
| ``` | ||
| git clone https://github.com/tektoncd/catalog.git | ||
| cd catalog/task/catlin-validate/0.1/tests | ||
| kubectl apply -f input-configmap.yaml | ||
| kubectl apply -f source-success-task-configmap.yaml | ||
| kubectl create -f taskrun-success.yaml | ||
| kubectl logs catlin-run-pod | ||
| ``` | ||
| * Pod log output | ||
| ```Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init) | ||
| -------catlin output-------- | ||
| FILE: /workspace/source/task/hello/0.1/hello.yaml | ||
|
|
||
| -------catlin output end-------- | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| ### Catlin Validate Warn | ||
| * Follow the below steps to run the sample taskrun in your local cluster which validates a task that has a warning. | ||
| * **Note** Warning will not fail the task. Task will still succeed but with [catlin-status](#catlin-status) as warning | ||
| ``` | ||
| git clone https://github.com/tektoncd/catalog.git | ||
| cd catalog/task/catlin-validate/0.1/tests | ||
| kubectl apply -f input-configmap.yaml | ||
| kubectl apply -f source-warn-task-configmap.yaml | ||
| kubectl create -f taskrun-warn.yaml | ||
| kubectl logs catlin-run-pod | ||
| ``` | ||
| * Pod log output | ||
| ```Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init) | ||
| -------catlin output-------- | ||
| FILE: /workspace/source/task/hello/0.1/hello.yaml | ||
| WARN : Step "echo" uses image "$(params.image)" that contains variables; skipping validation | ||
|
|
||
| -------catlin output end-------- | ||
| ``` | ||
| * Catlin output logs are also stored in [$(params.outputFile)](#outputfile) in [catlin-output](#catlin-output) workspace. This workspace can be passed on to other tasks in case other tasks need to consume this output. An example would be git-comment task can use this workspace to post the catlin output logs to the PR created to add a task to tekton catalog | ||
|
|
||
|
|
||
| ### Catlin Validate Error | ||
| * Follow the below steps to run the sample taskrun in your local cluster which validates a task that is non-conforming to the tekton-catalog standards | ||
| * **Note** Error will result in [catlin-status](#catlin-status) having the value as failure. | ||
| ``` | ||
| git clone https://github.com/tektoncd/catalog.git | ||
| cd catalog/task/catlin-validate/0.1/samples | ||
| kubectl apply -f input-configmap.yaml | ||
| kubectl apply -f source-error-task-configmap.yaml | ||
| kubectl create -f taskrun-error.yaml | ||
| kubectl logs catlin-run-pod | ||
| ``` | ||
| * Pod logs output | ||
| ``` | ||
| Defaulted container "step-catlin" out of: step-catlin, prepare (init), place-scripts (init), working-dir-initializer (init) | ||
| Error: /workspace/source/task/hello/0.1/hello.yaml failed validation | ||
| -------catlin output-------- | ||
| FILE: /workspace/source/task/hello/0.1/hello.yaml | ||
| ERROR: Resource path is invalid; expected path: task/hello/hello.yaml | ||
| ERROR: Task: tekton.dev/v1beta1 - name: "hello" must have a label "app.kubernetes.io/version" to indicate version | ||
| ERROR: Task: tekton.dev/v1beta1 - name: "hello" is missing a mandatory annotation for minimum pipeline version("tekton.dev/pipelines.minVersion") | ||
| ERROR: Task: tekton.dev/v1beta1 - name: "hello" is missing a mandatory annotation for category("tekton.dev/categories") | ||
| ERROR: Category not defined | ||
| You can choose from the categories present at location: https://raw.githubusercontent.com/tektoncd/hub/main/config.yaml" | ||
| HINT : Task: tekton.dev/v1beta1 - name: "hello" is missing a readable display name annotation("tekton.dev/displayName") | ||
| ERROR: Task: tekton.dev/v1beta1 - name: "hello" must have a description | ||
| HINT : Task: tekton.dev/v1beta1 - name: "hello" is easily discoverable if it has annotation for tag "tekton.dev/tags" | ||
| HINT : Task: tekton.dev/v1beta1 - name: "hello" is more usable if it has "tekton.dev/platforms" annotation about platforms to run | ||
| WARN : Step "echo" uses image "alpine"; consider using a fully qualified name - e.g. docker.io/library/ubuntu:1.0 | ||
| ERROR: Step "echo" uses image "alpine" which must be tagged with a specific version | ||
|
|
||
| -------catlin output end-------- | ||
| ``` | ||
|
|
||
| * Catlin output is also stored in [$(params.outputFile)](#outputfile) in [catlin-output](#catlin-output) workspace. This workspace can be passed on to other tasks in case other tasks need to consume this output. An example would be binding the output workspace to a workspace in git-comment to post the catlin output as a comment for a PR of a new task submission |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: Task | ||
| metadata: | ||
| name: catlin-validate | ||
| labels: | ||
| app.kubernetes.io/version: "0.1" | ||
| annotations: | ||
| tekton.dev/categories: Testing | ||
| tekton.dev/pipelines.minVersion: "0.17.0" | ||
| tekton.dev/tags: "tekton task yaml validation" | ||
| tekton.dev/platforms: "linux/amd64" | ||
| tekton.dev/displayName: catlin | ||
| spec: | ||
| description: >- | ||
| Catlin(https://github.com/tektoncd/catlin) is a command-line tool that Lints Tekton Resources and Catalogs. | ||
| Catlin Validate task supports running catlin validate for | ||
| 1. An isolated task yaml that you want to submit to tekton catalog | ||
| 2. A set of tasks that you want to submit to tekton catalog | ||
| params: | ||
| - name: outputFile | ||
| description: File that contains final catlin output. Catlin output will be stored in /workspaces/catlin-output/$(params.outputFile) | ||
| type: string | ||
| default: catlin-output.txt | ||
| - name: inputFile | ||
| description: | | ||
| File that contains catlin input. This file contains relative paths to the task yaml's delimited by newline | ||
| this file will be read from /workspaces/catlin-input/$(params.inputFile) and catlin validate will be applied to each file | ||
| example if file contains below paths (as you can see the paths are separated by newlines) | ||
| task/git-clone/0.9/git-clone.yaml | ||
| task/git-clone/0.1/git-comment.yaml | ||
| it will run catlin on the above paths in the source workspace one after another and store its output in catlin-output workspace | ||
| final command run will be | ||
| 'catlin validate $(workspaces.source.path)/task/git-clone/0.9/git-clone.yaml >> $(workspaces.catlin-output.path)/output.txt' | ||
| 'catlin validate $(workspaces.source.path)/task/git-clone/0.1/git-comment.yaml >> $(workspaces.catlin-output.path)/output.txt' | ||
| type: string | ||
| - name: ignoreWarnings | ||
| description: | | ||
| Ignores any warnings in catlin output. | ||
| If catlin output has only success and warnings, all warnings are ignored and final catlin output will be success | ||
| type: string | ||
| default: "false" | ||
| results: | ||
| - name: catlin-status | ||
| description: | | ||
| final status of catlin command. If all catlin validate command succeeds status will be set to 'success', if any one command validate command fails | ||
| status will be set to 'failure', If there is no failure and atleast one catline validate command is warning the status will be set to 'warning' | ||
| workspaces: | ||
| - name: source | ||
| description: | | ||
| This workspace contains files on which catlin will be run. task yamls in this folder should have a path that follows tekton-catalog folder structure. | ||
| example of folder structure: task/git-clone/0.9/git-clone.yaml | ||
| refer https://github.com/tektoncd/catalog for more details | ||
| - name: catlin-input | ||
| description: | | ||
| This workspace contains a file $(params.inputFile) delimited by newline containing paths to task yaml's on which catlin validate will be applied | ||
| - name: catlin-output | ||
| description: This workspace is where catlin output will be stored | ||
| steps: | ||
| - name: catlin-validate | ||
| image: gcr.io/tekton-releases/dogfooding/catlin:v20230721-0755082305@sha256:8db5e290b9a1b7794c536dbc826bb9cf3eeb82c41b251b7caee12204bcfb6664 | ||
| env: | ||
| - name: OUTPUT_FILE | ||
| value: $(params.outputFile) | ||
| - name: INPUT_FILE | ||
| value: $(params.inputFile) | ||
| - name: CATLIN_OUTPUT_WORKSPACE_PATH | ||
| value: $(workspaces.catlin-output.path) | ||
| - name: IGNORE_WARNINGS | ||
| value: $(params.ignoreWarnings) | ||
| workingDir: $(workspaces.catlin-input.path) | ||
| script: | | ||
| #!/usr/bin/env bash | ||
| rm -rf "${CATLIN_OUTPUT_WORKSPACE_PATH:?}"/"$OUTPUT_FILE" | ||
| while IFS="" read -r p || [ -n "$p" ] | ||
| do | ||
| catlin validate "$(workspaces.source.path)"/"$p" >> "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE" 2>&1 | ||
| echo "" >> "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE" | ||
| done < "$INPUT_FILE" | ||
|
|
||
| echo "" | ||
| echo "--------------------------------CATLIN OUTPUT START------------------------------------------" | ||
| echo "" | ||
| cat "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE" | ||
| echo "--------------------------------CATLIN OUTPUT END--------------------------------------------" | ||
| echo "" | ||
|
|
||
| STR=$(cat "$CATLIN_OUTPUT_WORKSPACE_PATH"/"$OUTPUT_FILE") | ||
| ERROR='ERROR' | ||
| WARN='WARN' | ||
| if [[ "$IGNORE_WARNINGS" == "true" ]]; then | ||
| echo "Ignore Warnings is set to true, any warnings in catlin output will be ignored" | ||
| fi | ||
|
|
||
| if [[ "$STR" == *"$ERROR"* || "$STR" == *"Error"* ]]; then | ||
| echo "failure" | tr -d "\n" > "$(results.catlin-status.path)" | ||
| echo "" | ||
| echo "Final output of catlin validate is failed since one of the files Errored out." | ||
| exit 1 | ||
| elif [[ "$STR" == *"$WARN"* && "$IGNORE_WARNINGS" == "false" ]]; then | ||
| echo "warning" | tr -d "\n" > "$(results.catlin-status.path)" | ||
| echo "" | ||
| echo "Final output of catlin validate is warning since one of the files has WARNING in its output." | ||
| else | ||
| echo "success" | tr -d "\n" > "$(results.catlin-status.path)" | ||
| echo "" | ||
| echo "Final output of catlin validate is success." | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: catlin-input | ||
| data: | ||
| input.txt: | | ||
| task/hello/0.1/hello.yaml |
17 changes: 17 additions & 0 deletions
17
task/catlin-validate/0.1/samples/source-error-task-configmap.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: source-error | ||
| data: | ||
| hello.yaml: | | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: Task | ||
| metadata: | ||
| name: hello | ||
| spec: | ||
| steps: | ||
| - name: echo | ||
| image: alpine | ||
| script: | | ||
| #!/bin/sh | ||
| echo "Hello World" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: TaskRun | ||
| metadata: | ||
| name: catlin-run-error | ||
| spec: | ||
| taskRef: | ||
| name: catlin-validate | ||
| params: | ||
| - name: inputFile | ||
| # file in catlin input workspace that contains newline delimited paths to task yaml files on which catlin validate will be applied | ||
| value: input.txt | ||
| workspaces: | ||
| - name: catlin-output | ||
| emptyDir: {} | ||
| - name: catlin-input | ||
| configmap: | ||
| name: catlin-input | ||
| - name: source | ||
| configmap: | ||
| name: source-error | ||
| items: | ||
| - key: "hello.yaml" | ||
| path: "task/hello/0.1/hello.yaml" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: catlin-input | ||
| data: | ||
| input.txt: | | ||
| task/hello/0.1/hello.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: catlin-input-multiple | ||
| data: | ||
| input.txt: | | ||
| task/hello/0.1/hello.yaml | ||
| task/bye/0.1/bye.yaml | ||
50 changes: 50 additions & 0 deletions
50
task/catlin-validate/0.1/tests/source-multiple-tasks-configmap.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: source-multiple | ||
| data: | ||
| hello.yaml: | | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: Task | ||
| metadata: | ||
| name: hello | ||
| labels: | ||
| app.kubernetes.io/version: "0.1" | ||
| annotations: | ||
| tekton.dev/categories: Testing | ||
| tekton.dev/pipelines.minVersion: "0.17.0" | ||
| tekton.dev/tags: "testing" | ||
| tekton.dev/platforms: "linux/amd64" | ||
| tekton.dev/displayName: hello | ||
| spec: | ||
| description: "hello task" | ||
| steps: | ||
| - name: echo | ||
| image: docker.io/library/ubuntu:18.04 | ||
| script: | | ||
| #!/bin/sh | ||
| echo "Hello World" | ||
| bye.yaml: | | ||
| apiVersion: tekton.dev/v1beta1 | ||
| kind: Task | ||
| metadata: | ||
| name: bye | ||
| labels: | ||
| app.kubernetes.io/version: "0.1" | ||
| annotations: | ||
| tekton.dev/categories: Testing | ||
| tekton.dev/pipelines.minVersion: "0.17.0" | ||
| tekton.dev/tags: "testing" | ||
| tekton.dev/platforms: "linux/amd64" | ||
| tekton.dev/displayName: bye | ||
| spec: | ||
| description: "bye task" | ||
| params: | ||
| - name: image | ||
| type: string | ||
| steps: | ||
| - name: echo | ||
| image: $(params.image) | ||
| script: | | ||
| #!/bin/sh | ||
| echo "bye" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all these can be passed using parameters of type array. Can you please explain the exact usecase of mounting a configmap for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have used a configmap since the input to the task is a file in a workspace instead of array params. This (#1178 (comment)) is the reason to keep it a file in a workspace rather than input parameter array.