Independent container updates #1080

almet · 2025-02-11T18:50:56Z

This pull request introduces independent container updates.

Signatures are meant to be done with an hardware key (yubikey), for which the public key will be packaged alongside the code.

container signatures are stored locally and checked against a known public key just before doing the conversion of the documents. Behind the curtain, it uses cosign so that the signatures are publicly auditable.

It adds a dangerzone-image CLI providing tooling to check for remote new images, verification of the attestations and signatures, also with the ability to create and use archives, for air-gapped environments.

$ dangerzone-image --help

  attest-provenance  Look up the image attestation to see if the image...
  get-manifest       Retrieves a remote manifest for a given image and...
  list-remote-tags   List the tags available for a given image.
  load-archive       Upgrade the local image to the one in the archive.
  prepare-archive    Prepare an archive to upgrade the dangerzone image...
  upgrade            Upgrade the image to the latest signed version.
  verify-local       Verify the local image signature against a public...

It is fixing the following issues:

The last commit here introduces a way to use the code, but is not meant to be shipped "as is". We still need to get the full story for this, especially in terms of UX.

Before being ready to be shipped, we still miss a few key components:

Fix the linter
Implement unit tests
- signatures
- registry
Publish and sign test images / use them in tests
Package/vendorize cosign (Package/vendorize cosign #1062)
Implement the UI, related to the discussion on Clarify the UX for Independent Container Updates #1079
Clean after ourselves. We're changing the way we name the container, but we should probably remove the old images.

It contains utilities to interact with OCI registries, like getting the list of published tags and getting the content of a manifest. It does so via the use of the Docker Registry API v2 [0]. The script has been added to the `dev_scripts`, and is also installed on the system under `dangerzone-image`. [0] https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

Signatures are stored in the OCI Manifest v2 registry [0], and are expected to follow the Cosign Signature Specification [0] The following CLI utilities are provided with `dangerzone-image`: For checking new container images, upgrading them and downloading them: - `upgrade` allows to upgrade the current installed image to the last one available on the OCI registry, downloading and storing the signatures in the process. - `verify-local` allows the verify the currently installed image against downloaded signatures and public key. To prepare and install archives on air-gapped environments: - `prepare-archive` helps to prepare an archive to install on another machine - `load-archive` helps upgrade the local image to the archive given in argument. Signatures are stored locally using the format provided by `cosign download signature`, and the Rekor log index is used to ensure the requested-to-install container image is fresher than the one already present on the system. [0] https://github.com/sigstore/cosign/blob/main/specs/SIGNATURE_SPEC.md

The hash list provided on the Github releases page is now bundled in the `reproduce-image.py` script, and the proper hashes are checked after download.

A new `dangerzone-image attest-provenance` script is now available, making it possible to verify the attestations of an image published on the github container registry. Container images are now build nightly and uploaded to the container registry.

…txt`

apyrgio · 2025-02-18T14:02:12Z

.github/workflows/release-container-image.yml

Heads up, I'll make changes in this file, based on the findings in https://github.com/freedomofpress/repro-build. Nothing too radical, the changes in our build-image.py script will be far more reaching. I'll add those as new commits, if you don't mind.

apyrgio · 2025-02-18T14:11:06Z

dangerzone/container_utils.py

-
-
-def load_image_tarball() -> None:
+def load_image_tarball_from_gzip() -> None:


While working on reproducible builds, I started to realize that our container image becomes marginally smaller if we compress it with gzip. That could be because Buildkit compresses the files in each layer, but I'm not sure yet. Anyway, what I'm getting at is that this code path is a strong candidate for ✂️ .

apyrgio · 2025-02-18T14:19:18Z

dangerzone/container_utils.py

+
+def container_pull(image: str) -> bool:
+    """Pull a container image from a registry."""
+    cmd = [get_runtime_name(), "pull", f"{image}"]


Suggested change

cmd = [get_runtime_name(), "pull", f"{image}"]

cmd = [get_runtime_name(), "pull", image]

apyrgio · 2025-02-18T14:23:31Z

dangerzone/container_utils.py

+    process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+    process.communicate()


What do we want to achieve with .communicate(). Is it to enable some sort of progress report in the future? In any case, I think a progress_fn and iterating the stdout lines will do.

apyrgio · 2025-02-18T14:27:42Z

dangerzone/container_utils.py

+        if not image_digest:
+            raise errors.ImageNotPresentException(
+                f"The image {image} does not exist locally"
+            )


If I get what you're talking about, you're handling here the case where the image exists locally, but Podman/Docker somehow don't return a digest for it. That's a weird one indeed, I've seen it while testing things out, but never could figure out why it happens.

At any rate, I'd change the message here to show that the image exists locally, but the container engine reports no digest for it. Solely to not get confused with the error message below.

apyrgio · 2025-02-18T14:54:12Z

dangerzone/isolation_provider/container.py

+        if False:  # Comment this for now, just as an exemple of this can be implemented
+            # # Load the image tarball into the container runtime.
+            update_available, image_digest = updater.is_update_available(
+                container_utils.CONTAINER_NAME


Keeping a note here to revisit this method once we nail the UX.

apyrgio · 2025-02-18T15:03:47Z

dangerzone/updater/attestations.py

+# NOTE: You can grab the SLSA attestation for an image/tag pair with the following
+# commands:
+#
+#     IMAGE=ghcr.io/apyrgio/dangerzone/dangerzone
+#     TAG=20250129-0.8.0-149-gbf2f5ac
+#     DIGEST=$(crane digest ${IMAGE?}:${TAG?})
+#     ATT_MANIFEST=${IMAGE?}:${DIGEST/:/-}.att
+#     ATT_BLOB=${IMAGE?}@$(crane manifest ${ATT_MANIFEST?} | jq -r '.layers[0].digest')
+#     crane blob ${ATT_BLOB?} | jq -r '.payload' | base64 -d | jq
+CUE_POLICY = r"""


Before settling on the final CUE policy, I'd like to take one last look at the SLSA attestations, to see if we can use any of them.

apyrgio · 2025-02-18T15:39:23Z

dangerzone/updater/signatures.py

+    return False
+
+
+def is_update_available(image: str) -> Tuple[bool, Optional[str]]:


I have a few comments on this function:

Ideally, I'd prefer if the log index check happened here, so that this function never returns True if we can't perform the update.

I'm not sure if this function is related with signatures. Should it go to a different module, perhaps the registry.py one?

If the image name already contains a digest, we can use that instead of the network call.

apyrgio · 2025-02-18T16:14:02Z

dangerzone/updater/signatures.py

+        log.debug(" ".join(cmd))
+        result = subprocess.run(cmd, capture_output=True)
+        if result.returncode != 0:
+            # XXX Raise instead?


I agree, we already raise in the above lines, so I wouldn't mix booleans here.

apyrgio · 2025-02-18T16:19:23Z

dangerzone/updater/signatures.py

+    }
+
+
+def verify_signature(signature: dict, image_digest: str, pubkey: str | Path) -> bool:


Minor, but it's kind of a sensitive function, so I want to be more explicit about what we verify here. I'd call this verify_image_digest, because it verifies that the digest of a container image has been signed by our private key. If you're ok with it, let's update the docstring as well.

almet and others added 6 commits February 11, 2025 18:13

Add the ability to download diffoci for multiple platforms

197325b

The hash list provided on the Github releases page is now bundled in the `reproduce-image.py` script, and the proper hashes are checked after download.

Add documentation for independent container updates

af55d26

(WIP) Check for container updates rather than using image-id.txt

5c9a38d

almet mentioned this pull request Feb 11, 2025

Publish image attestations #1017

Closed

almet added 23 commits February 12, 2025 11:40

fixup! Download and verify cosign signatures

27647cc

fixup! Download and verify cosign signatures

f6562ae

fixup! Download and verify cosign signatures

7002ab8

fixup! Download and verify cosign signatures

6aff845

fixup! Download and verify cosign signatures

db33038

fixup! Download and verify cosign signatures

5001328

fixup! Download and verify cosign signatures

22d235c

fixup! Download and verify cosign signatures

5a4ddb1

fixup! Download and verify cosign signatures

1e9e468

fixup! Add a dangerzone-image CLI script

379c9f8

fixup! Add a dangerzone-image CLI script

d667c28

fixup! Add a dangerzone-image CLI script

431f0cb

fixup! Add a dangerzone-image CLI script

aac6c63

fixup! Add a dangerzone-image CLI script

ccae6c5

fixup! Add a dangerzone-image CLI script

5202d79

fixup! Add a dangerzone-image CLI script

9889710

fixup! Add a dangerzone-image CLI script

668ee71

fixup! Publish and attest multi-architecture container images

0724f86

fixup! Publish and attest multi-architecture container images

537d23e

fixup! Publish and attest multi-architecture container images

5acb302

fixup! 1e9e468

e078e9b

fixup! (WIP) Check for container updates rather than using `image-id.…

60674ea

…txt`

fixup! (WIP) Check for container updates rather than using `image-id.…

835970b

…txt`

almet added 2 commits February 12, 2025 18:23

(WIP) Add tests

a540fc5

(WIP) some more tests

0f2d81d

apyrgio reviewed Feb 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Independent container updates #1080

Independent container updates #1080

almet commented Feb 11, 2025 •

edited

Loading

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025

apyrgio Feb 18, 2025



		def load_image_tarball() -> None:
		def load_image_tarball_from_gzip() -> None:

	cmd = [get_runtime_name(), "pull", f"{image}"]
	cmd = [get_runtime_name(), "pull", image]

		process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
		process.communicate()

		return False


		def is_update_available(image: str) -> Tuple[bool, Optional[str]]:

		}


		def verify_signature(signature: dict, image_digest: str, pubkey: str \| Path) -> bool:

Independent container updates #1080

Are you sure you want to change the base?

Independent container updates #1080

Conversation

almet commented Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

almet commented Feb 11, 2025 •

edited

Loading