Add sbom scanning command

[LaurentGoderre]

This adds a command to generate SBOM independently of the build which has several advantages:

• Unified approach for generating SBOM across different building methods (buildkit, OCI IMport etc..)
• Adds ability to support more architectures by decoupling the build and the scan architecture
• Ability to fix broken or out of date SBOM without needing to trigger a new build.

[tianon]

No description provided.

I know we talked about this project offline, but nobody outside you and I were part of that conversation and it's not stored anywhere, so the PR description for something like this is a good place to expound on the plan/purpose we discussed. 😅

It's also an important place to put into context which part of what we discussed this is, and why adding it here in the production repository now (without all the rest of what we talked about in place/ready) helps or unblocks you. 🙇 ❤️

For example:

Offline, we've discussed trying to separate SBOM generation from build, especially so we can regenerate an SBOM without rebuilding an image (new/better/more accurate SBOM scanner, for example).

This PR adds a new "command" in meta.jq for generating an SBOM of an image that's already built (and thus .build.resolved in builds.json is non-null).

Nothing is using this yet, but adding it here now makes it easier for me to XXX

Where "XXX" is information I don't currently have -- I have some technical/nit review comments I can make about this PR, but without that broader context telling me how this PR fits into that larger picture (and why it's being submitted by itself), I'm likely missing important information about why you've made the choices you did in the implementation you've proposed. 🙈 ❤️

[tianon]

This adds a command to generate SBOM independently of the build which has several advantages:

• Unified approach for generating SBOM across different building methods (buildkit, OCI IMport etc..)
• Adds ability to support more architectures by decoupling the build and the scan architecture
• Ability to fix broken or out of date SBOM without needing to trigger a new build.

Sorry, I just noticed you updated the description (there's no GitHub notification for that, nor entry in the event log on the PR 😭).

This is really good "what", but I'm still missing the "why" for this PR's changes specifically -- ie, if we merge this and commit to maintain it here, what does that unlock for you? This isn't the full solution to generating SBOMs in a detached way, and as we've discussed that's going to be a much larger project that needs to involve the full lifecycle management, so as-is this is going to be unused code for us for now (and will continue to be for the near-term future), so I'm missing what proposing this change here does beyond adding more red tape/review for your PoC/experimentation work? Can you please elaborate on that?

For context, when I'm making larger interconnected changes like this and working on proving them out, I'll use a forked branch that adds my changes and rebase it periodically while I continue experimenting and integrating elsewhere, patching the other parts to point at my forked copy in the experimentation phases. In this specific case, that might look like the GitHub workflow you're working with doing actions/checkout against docker-library/meta, and then an explicit run: step (or another actions/checkout, if you wanted to be fancy) that checks out your forked version of docker-library/meta-scripts at the correct branch on top of meta's .scripts directory in order to test the complete integration end-to-end.

Does that help explain what I'm looking for / missing better? 🙇 ❤️

[LaurentGoderre]

Bringing the code in earlier allows us to parallelize the different parts of the process instead of doing it linearly and would be very similar to how feature flagging works.

Additionally, this would allow proving that large portions of the code for signing and creating sbom etc work even if the final workflows are not in place. We can then create pipelines in a fork that use the production scripts generating SBOMs for production images but sending them to a seperate destination.

Note also that I also am providing two possible approach we can take for this.