root-signing metadata is incompatible with current sigstore-rs

[jku]

This is something that came up during staging testing: sigstore-rs is not compatible with root-signing-staging, and will not be compatible with root-signing if we proceed with #929 without changes.

• Current root-signing metadata contains metadata hashes and lengths, but tuf-on-ci produces metadata that does not contain them
• both variants are spec compliant
• awslabs/tough used by sigstore-rs does not currently support the tuf-on-ci produced metadata
• sigstore-rs is experimental and does not have releases so was not included in the root-signing staging test matrix so the issue was not noticed earlier
• there is a related compatibility problem with keyids: this is not an issue in root-signing and will be fixed in root-signing-staging

I'm filing this so we can decide whether this is a blocker for #929 or not. I would suggest it's not a blocker:

• the tuf-on-ci metadata is compliant wrt hashes and lengths
• adding support for this in the client (awslabs/tough) should not be a major issue

That said, tuf-on-ci could start embedding hashes and lengths if that is really needed.

Related sigstore-rs issue sigstore/sigstore-rs#369

[haydentherapper]

Do you know if https://github.com/theupdateframework/rust-tuf would be compatible or is maintained more actively?

[jku]

IIRC they don't have a CLI so testing would be a bit more work (this specific part of the spec seems to be supported but that doesn't mean much)

[jku]

• I believe the upstream PR fixing the blocker has been merged in awslabs/tough: fix: timestamp.json meta can has optional fields awslabs/tough#778
• awslabs/tough release, update in sigstore-rs and a sigstore-rs release would still be needed
• we still don't have an automated test for sigstore-rs in root-signing (-staging) so it's not 100% confirmed that sigstore-rs is then compatible... but one would be welcome in https://github.com/sigstore/root-signing-staging/blob/main/.github/workflows/custom-test.yml

[jku]

the metadata in question is now published

[jku]

for the record awslabs/tough has released... but now there is a hairy dependency deadlock that still prevents sigstore-rs from using the new release.

[jku]

I believe this has been fixed with the latest sigstore-rs release