Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: save containerd image into archive and use in tests #7816

Merged

Conversation

DmitriyLewen
Copy link
Contributor

@DmitriyLewen DmitriyLewen commented Oct 29, 2024

Description

To avoid downloading containerd image from remote registry for each containerd test, we need:

  • mage test:FixtureContainerImages command should download containerd image as archive.
  • load containerd image from archive for containerd tests
  • update digest for cache in GitHub Actions

test run - https://github.com/aquasecurity/trivy/actions/runs/11572399371/job/32212304048?pr=7816#step:7:1716

Related issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@DmitriyLewen DmitriyLewen self-assigned this Oct 29, 2024
@DmitriyLewen DmitriyLewen marked this pull request as ready for review October 29, 2024 11:10
@knqyf263
Copy link
Collaborator

@DmitriyLewen Do you think we should change ghcr.io/aquasecurity/trivy-test-images/containerd to ghcr.io/aquasecurity/trivy-test-images:containerd?

@DmitriyLewen
Copy link
Contributor Author

Yesterday after work I also thought about this.
There is difference between containers:
trivy-test-images are the target for scanning, and trivy-test-images/containerd is an auxiliary image (for example, like a registry)
from the point of view of order - combining all images is not entirely correct, but combining will help unify and simplify interaction with test containers.

I still think that we shouldn't merge them.
If we have more similar images, we'll think about this issue again (although it would probably be more correct to combine the targets into one image, and the auxiliary images into another)

@@ -29,7 +29,7 @@ jobs:
run: |
source integration/testimages.ini
IMAGE_LIST=$(skopeo list-tags docker://$TEST_IMAGES)
DIGEST=$(echo "$IMAGE_LIST" | sha256sum | cut -d' ' -f1)
DIGEST=$(echo "$IMAGE_LIST" | jq '.Tags += ["containerd"]' | sha256sum | cut -d' ' -f1)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized the order depends on the registry. I think we should sort the image list.

$ crane ls public.ecr.aws/aquasecurity/trivy-test-images
centos-6
almalinux-8
photon-30
sle-micro-rancher-5.4_ndb
alpine-310
alpine-distroless
fluentd-multiple-lockfiles
debian-stretch
centos-7
ubi-7
debian-buster
distroless-python27
opensuse-leap-151
spring4shell-jre8
ubuntu-1604
distroless-base
fedora-35
mariner-1.0
busybox-with-lockfile
alpine-39
ubuntu-1804
vulnimage
amazon-2
oraclelinux-8
rockylinux-8
amazon-1
suse-15.3_ndb
opensuse-tumbleweed
spring4shell-jre11

Copy link
Contributor Author

@DmitriyLewen DmitriyLewen Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

I also found, that skopeo list-tags returns repository name:

# skopeo list-tags docker://ghcr.io/knqyf263/trivy-test-images
{
    "Repository": "ghcr.io/knqyf263/trivy-test-images",
    "Tags": [
        "spring4shell-jre11",
        "spring4shell-jre8",
        "almalinux-8",
        "alpine-39",
        "alpine-310",
        "alpine-distroless",
        "busybox-with-lockfile",
        "fluentd-multiple-lockfiles",
        "amazon-1",
        "amazon-2",
        "centos-6",
        "centos-7",
        "debian-buster",
        "debian-stretch",
        "distroless-base",
        "distroless-python27",
        "mariner-1.0",
        "opensuse-leap-151",
        "photon-30",
        "rockylinux-8",
        "ubi-7",
        "ubuntu-1604",
        "ubuntu-1804",
        "vulnimage",
        "oraclelinux-8",
        "suse-15.3_ndb",
        "fedora-35",
        "opensuse-tumbleweed",
        "sle-micro-rancher-5.4_ndb"
    ]
}

Therefore, to calculate the digest I take only sorted tags (see f478633):

root@73c3501e9402:/scripts# TEST_IMAGES=public.ecr.aws/aquasecurity/trivy-test-images ./script.sh 
digest=d8b4d51584e5d09e38687eb50c7d6a2ff49d57c28ff14e3d04cd22d9c8175354
root@73c3501e9402:/scripts# TEST_IMAGES=ghcr.io/knqyf263/trivy-test-images ./script.sh 
digest=d8b4d51584e5d09e38687eb50c7d6a2ff49d57c28ff14e3d04cd22d9c8175354

@knqyf263
Copy link
Collaborator

We most likely need to merge the main branch.

@DmitriyLewen
Copy link
Contributor Author

Thanks. Merged.

@knqyf263 knqyf263 added this pull request to the merge queue Nov 4, 2024
Merged via the queue into aquasecurity:main with commit a6a45b3 Nov 4, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test: cache containerd image
2 participants