Skip to content

Commit

Permalink
Combine all FBC validation tasks into one
Browse files Browse the repository at this point in the history
This change combines the inspect-image, fbc-validate, and
fbc-related-image-check into a single task.

All of the old tasks will be deprecated and the required tasks need to
be updated to require only

```
- [fbc-related-image-check, validate-fbc]
```

This will ensure that users can still be guaranteed to have a valid FBC
fragment and appropriate related images.

The following items were deferred from this task update:
* make this task multi-arch aware.
* make this task fail if a bundle is an image index reference
* change the related-image check to be done in EC by exporting the set of related images

Signed-off-by: arewm <[email protected]>
  • Loading branch information
arewm committed Nov 19, 2024
1 parent 6f4a50d commit 6f9f4e1
Show file tree
Hide file tree
Showing 23 changed files with 869 additions and 82 deletions.
5 changes: 5 additions & 0 deletions hack/missing-ta-tasks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,14 @@ todo=(
task/buildah-rhtap/0.1/buildah-rhtap.yaml
task/download-sbom-from-url-in-attestation/0.1/download-sbom-from-url-in-attestation.yaml
task/fbc-related-image-check/0.1/fbc-related-image-check.yaml
task/fbc-related-image-check/0.2/kustomization.yaml
task/fbc-validation/0.1/fbc-validation.yaml
task/fbc-validation/0.2/kustomization.yaml
task/gather-deploy-images/0.1/gather-deploy-images.yaml
task/generate-odcs-compose/0.2/generate-odcs-compose.yaml
task/generate-odcs-compose/0.2/kustomization.yaml
task/inspect-image/0.1/inspect-image.yaml
task/inspect-image/0.2/kustomization.yaml
task/operator-sdk-generate-bundle/0.1/operator-sdk-generate-bundle.yaml
task/opm-get-bundle-version/0.1/opm-get-bundle-version.yaml
task/opm-render-bundles/0.1/opm-render-bundles.yaml
Expand All @@ -50,6 +53,8 @@ emit() {
fi
}

echo ${todo[@]}

{
cd "${git_root}"
missing=0
Expand Down
49 changes: 11 additions & 38 deletions pipelines/fbc-builder/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,6 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE_URL| Fully qualified image name.| None| '$(tasks.build-image-index.results.IMAGE_URL)'|
|POLICY_DIR| Path to directory containing Conftest policies.| /project/repository/| |
|POLICY_NAMESPACE| Namespace for Conftest policy.| required_checks| |
### fbc-validation:0.1 task parameters
|name|description|default value|already set by|
|---|---|---|---|
|BASE_IMAGE| Fully qualified base image name.| None| '$(tasks.inspect-image.results.BASE_IMAGE)'|
|IMAGE_DIGEST| Image digest.| None| '$(tasks.build-image-index.results.IMAGE_DIGEST)'|
|IMAGE_URL| Fully qualified image name.| None| '$(tasks.build-image-index.results.IMAGE_URL)'|
### git-clone:0.1 task parameters
|name|description|default value|already set by|
|---|---|---|---|
Expand Down Expand Up @@ -110,12 +104,6 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|image-url| Image URL for build by PipelineRun| None| '$(params.output-image)'|
|rebuild| Rebuild the image if exists| false| '$(params.rebuild)'|
|skip-checks| Skip checks against built image| false| '$(params.skip-checks)'|
### inspect-image:0.1 task parameters
|name|description|default value|already set by|
|---|---|---|---|
|DOCKER_AUTH| unused, should be removed in next task version| | |
|IMAGE_DIGEST| Image digest.| None| '$(tasks.build-image-index.results.IMAGE_DIGEST)'|
|IMAGE_URL| Fully qualified image name.| None| '$(tasks.build-image-index.results.IMAGE_URL)'|
### show-sbom:0.1 task parameters
|name|description|default value|already set by|
|---|---|---|---|
Expand All @@ -130,6 +118,11 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|git-url| Git URL| None| '$(tasks.clone-repository.results.url)?rev=$(tasks.clone-repository.results.commit)'|
|image-url| Image URL| None| '$(params.output-image)'|
|pipelinerun-name| pipeline-run to annotate| None| '$(context.pipelineRun.name)'|
### validate-fbc:0.1 task parameters
|name|description|default value|already set by|
|---|---|---|---|
|IMAGE_DIGEST| Image digest.| None| '$(tasks.build-image-index.results.IMAGE_DIGEST)'|
|IMAGE_URL| Fully qualified image name.| None| '$(tasks.build-image-index.results.IMAGE_URL)'|

## Results
|name|description|value|
Expand All @@ -143,9 +136,9 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
|IMAGES| List of all referenced image manifests| |
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; inspect-image:0.1:IMAGE_DIGEST ; fbc-validate:0.1:IMAGE_DIGEST|
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; validate-fbc:0.1:IMAGE_DIGEST|
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; apply-tags:0.1:IMAGE ; inspect-image:0.1:IMAGE_URL ; fbc-validate:0.1:IMAGE_URL|
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; apply-tags:0.1:IMAGE ; validate-fbc:0.1:IMAGE_URL|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### buildah:0.2 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
Expand All @@ -161,14 +154,6 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|---|---|---|
|IMAGES_PROCESSED| Images processed in the task.| |
|TEST_OUTPUT| Tekton task test output.| |
### fbc-related-image-check:0.1 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
|TEST_OUTPUT| Tekton task test output.| |
### fbc-validation:0.1 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
|TEST_OUTPUT| Tekton task test output.| |
### git-clone:0.1 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand All @@ -182,42 +167,30 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
|build| Defines if the image in param image-url should be built| |
### inspect-image:0.1 task results
### validate-fbc:0.1 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
|BASE_IMAGE| Base image source image is built from.| fbc-validate:0.1:BASE_IMAGE|
|BASE_IMAGE_REPOSITORY| Base image repository URL.| |
|RELATED_IMAGE_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the related images for the FBC fragment.| |
|TEST_OUTPUT| Tekton task test output.| |
|TEST_OUTPUT_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the related images for the FBC fragment.| |

## Workspaces
|name|description|optional|used in tasks
|---|---|---|---|
|git-auth| |True| clone-repository:0.1:basic-auth|
|netrc| |True| |
|workspace| |False| show-summary:0.2:workspace ; clone-repository:0.1:output ; build-container:0.2:source ; inspect-image:0.1:source ; fbc-validate:0.1:workspace ; fbc-related-image-check:0.1:workspace|
|workspace| |False| show-summary:0.2:workspace ; clone-repository:0.1:output ; build-container:0.2:source|
## Available workspaces from tasks
### buildah:0.2 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
|source| Workspace containing the source code to build.| False| workspace|
### fbc-related-image-check:0.1 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
|workspace| | False| workspace|
### fbc-validation:0.1 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
|workspace| | False| workspace|
### git-clone:0.1 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
|basic-auth| A Workspace containing a .gitconfig and .git-credentials file or username and password. These will be copied to the user's home before any git commands are run. Any other files in this Workspace are ignored. It is strongly recommended to use ssh-directory over basic-auth whenever possible and to bind a Secret to this Workspace over other volume types. | True| git-auth|
|output| The git repo will be cloned onto the volume backing this Workspace.| False| workspace|
|ssh-directory| A .ssh directory with private key, known_hosts, config, etc. Copied to the user's home before git commands are executed. Used to authenticate with the git remote when performing the clone. Binding a Secret to this Workspace is strongly recommended over other volume types. | True| |
### inspect-image:0.1 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
|source| | False| workspace|
### summary:0.2 task workspaces
|name|description|optional|workspace from pipeline
|---|---|---|---|
Expand Down
46 changes: 2 additions & 44 deletions pipelines/fbc-builder/patch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,60 +76,18 @@
- op: add
path: /spec/tasks/-
value:
name: inspect-image
name: validate-fbc
when:
- input: $(params.skip-checks)
operator: in
values: ["false"]
runAfter:
- build-image-index
taskRef:
name: inspect-image
name: validate-fbc
version: "0.1"
params:
- name: IMAGE_URL
value: $(tasks.build-image-index.results.IMAGE_URL)
- name: IMAGE_DIGEST
value: $(tasks.build-image-index.results.IMAGE_DIGEST)
workspaces:
- name: source
workspace: workspace
- op: add
path: /spec/tasks/-
value:
name: fbc-validate
when:
- input: $(params.skip-checks)
operator: in
values: ["false"]
runAfter:
- inspect-image
taskRef:
name: fbc-validation
version: "0.1"
params:
- name: IMAGE_URL
value: $(tasks.build-image-index.results.IMAGE_URL)
- name: IMAGE_DIGEST
value: $(tasks.build-image-index.results.IMAGE_DIGEST)
- name: BASE_IMAGE
value: $(tasks.inspect-image.results.BASE_IMAGE)
workspaces:
- name: workspace
workspace: workspace
- op: add
path: /spec/tasks/-
value:
name: fbc-related-image-check
when:
- input: $(params.skip-checks)
operator: in
values: ["false"]
runAfter:
- fbc-validate
taskRef:
name: fbc-related-image-check
version: "0.1"
workspaces:
- name: workspace
workspace: workspace
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ metadata:
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/tags: "konflux"
build.appstudio.redhat.com/expires-on: "2024-12-31T00:00:00Z"
name: fbc-related-image-check
spec:
description: >-
Expand Down
5 changes: 5 additions & 0 deletions task/fbc-related-image-check/0.1/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- fbc-related-image-check.yaml
48 changes: 48 additions & 0 deletions task/fbc-related-image-check/0.2/MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
## Deprecation notice

This task is deprecated, please remove it from your pipeline.
Deprecation date: 2024-12-31

# Migration from 0.1 to 0.2

Version 0.2:

No changes within this version, its only purpose is to provide information on how to remove this task from your pipeline.

## Action from users

To remove this task from your pipeline please follow these steps:

1. Remove the fbc-related-image-check task definition from your FBC pipelines similar to this change:

```diff
--- a/.tekton/original-pipelinerun.yaml
+++ b/.tekton/new-pipelinerun.yaml
@@ -323,26 +323,6 @@ spec:
workspaces:
- name: workspace
workspace: workspace
- - name: fbc-related-image-check
- runAfter:
- - fbc-validate
- taskRef:
- params:
- - name: name
- value: fbc-related-image-check
- - name: bundle
- value: quay.io/konflux-ci/tekton-catalog/task-fbc-related-image-check:0.1@sha256:0fae84cc832d21c250334ab1d285db92e7e22e916ea342d044e46136c502d2f8
- - name: kind
- value: task
- resolver: bundles
- when:
- - input: $(params.skip-checks)
- operator: in
- values:
- - "false"
- workspaces:
- - name: workspace
- workspace: workspace
workspaces:
- name: workspace
- name: git-auth
```
18 changes: 18 additions & 0 deletions task/fbc-related-image-check/0.2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# fbc-related-image-check task

## Description:
The fbc-related-image-check task checks whether all images referenced in file-based catalog (FBC) are valid by using
Skopeo to inspect manifest content.

## Results:

| name | description |
|-------------------|---------------------------|
| TEST_OUTPUT | Tekton task test output. |

## Source repository for image:
https://github.com/konflux-ci/konflux-test

## Additional links:
https://www.redhat.com/en/topics/containers/what-is-skopeo
https://olm.operatorframework.io/docs/reference/file-based-catalogs/
15 changes: 15 additions & 0 deletions task/fbc-related-image-check/0.2/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: kustomize.config.k8s.io/v1beta1

Check warning on line 1 in task/fbc-related-image-check/0.2/kustomization.yaml

View workflow job for this annotation

GitHub Actions / Check Trusted Artifact variants

TODO: Task needs a Trusted Artifacts variant created
kind: Kustomization

resources:
- ../0.1

patches:
- patch: |-
- op: replace
path: /metadata/labels
value:
app.kubernetes.io/version: "0.2"
target:
kind: Task
name: fbc-related-image-check
1 change: 1 addition & 0 deletions task/fbc-validation/0.1/fbc-validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ metadata:
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/tags: "konflux"
build.appstudio.redhat.com/expires-on: "2024-12-31T00:00:00Z"
name: fbc-validation
spec:
description: >-
Expand Down
5 changes: 5 additions & 0 deletions task/fbc-validation/0.1/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- fbc-validation.yaml
51 changes: 51 additions & 0 deletions task/fbc-validation/0.2/MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
## Deprecation notice

This task is deprecated, please remove it from your pipeline and replace it with the new validate-fbc task.
Deprecation date: 2024-12-31

# Migration from 0.1 to 0.2

Version 0.2:

No changes within this version, its only purpose is to provide information on how to remove this task from your pipeline.

## Action from users

To remove this task from your pipeline please follow these steps:

1. Remove the fbc-validation task definition from your FBC pipelines similar to this change:

```diff
--- a/.tekton/original-pipelinerun.yaml
+++ b/.tekton/new-pipelinerun.yaml
@@ -323,26 +323,6 @@ spec:
workspaces:
- name: workspace
workspace: workspace
- - name: fbc-validation
+ - name: validate-fbc
- runAfter:
- - inspect-image
+ - build-image-index
taskRef:
params:
- name: name
- value: fbc-validation
+ value: validate-fbc
- - name: bundle
- value: quay.io/konflux-ci/tekton-catalog/task-validate-fbc:0.1
- name: kind
value: task
resolver: bundles
when:
- input: $(params.skip-checks)
operator: in
values:
- "false"
- workspaces:
- - name: workspace
- workspace: workspace
workspaces:
- name: workspace
- name: git-auth
```
30 changes: 30 additions & 0 deletions task/fbc-validation/0.2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# fbc-validation task

## Description:
Ensures file-based catalog (FBC) components are uniquely linted for proper construction as part of build pipeline.

For further information on how to use the task, see the USAGE.md file.

For troubleshooting assistance, see the TROUBLESHOOTING.md file.

## Params:

| name | description |
|--------------|----------------------------------|
| IMAGE_DIGEST | Image digest. |
| IMAGE_URL | Fully qualified image name. |
| BASE_IMAGE | Fully qualified base image name. |

## Results:

| name | description |
|--------------------|---------------------------|
| TEST_OUTPUT | Tekton task test output. |

## Source repository for image:
https://github.com/konflux-ci/konflux-test

## Additional links:
https://olm.operatorframework.io/docs/reference/file-based-catalogs/
https://github.com/containers/skopeo
https://docs.openshift.com/container-platform/4.12/cli_reference/opm/cli-opm-install.html
Loading

0 comments on commit 6f9f4e1

Please sign in to comment.