Backwards Compatibility (TAPs 3, 4, 5)

[ecordell]

There's some hand-waving in the current TAPs about backwards-compatibility that I'd like to address. I think that backwards-compatibility is an important goal that would aid adoption, not just of TAPs but of TUF in general, by setting the precedent that the code you deploy today will continue to work tomorrow for some reasonable definition of tomorrow.

I think that the TAPs should analyze the pre- and post-TAP world for both client and server, and determine if it is safe to change the TAP to be backwards compatible. A client failure would be appropriate in situations where the feature provided by the TAP is in use and critical for valid client behavior.

TAP 3

Current Backwards Compatibility Section:

This TAP is incompatible with previous implementations of TUF because the targets metadata file format has been changed in a backwards-incompatible manner. However, note that it should take very little effort to adapt an existing implementation to resolve or encode delegations using the new file format.

New Backwards Compatibility Section Proposal:

This TAP can be made backwards compatible with the previous implementation of TUF by treating metadata with a name field the same as a metadata with a names field containing one entry.

Unaware clients (that is, clients that don't understand TAP3's changes) will fail to update to new metadata containing the new names key and not the name key.

Aware clients that attempt to update to new metadata containing the pre-TAP3 format will understand it as the degenerate case of TAP3's format and proceed as normal.

Because TAP3 does not address any security flaws, it is safe to allow old and new clients to coexist for a time.

TAP 4

Current Backwards Compatibility Section:

This specification is not backwards-compatible because it requires:

• TUF clients to support additional, optional fields in the root metadata file.
• A repository to use a specific filesystem layout.
• A client to use a map file.
• A client to use a specific filesystem layout.
• A client to download metadata and target files from a repository in a specific manner.

New Backwards Compatibility Section Proposal:

This TAP can be made backwards compatible with the previous implementation of TUF by treating a client without a mapping.json file as one with a single entry that delegates * to the root URL configured in the TUF client.

Unaware clients (that is, clients that don't understand TAP4's changes) will not have the ability to distribute trust in the way TAP4 specifies, but will continue to work assuming all requested targets live in the single configured repository.

Aware clients that attempt to update, but don't have a mapping file, will continue to operate the same as unaware clients. Aware clients that attempt to update and have a mapping file by definition operate as specified in this TAP.

There may be a case where a user assumes that an Unaware client is actually an Aware client, in which the user believes a mapping file is being consumed but is in fact being ignored. In most useful cases, the user will become aware of the issue because they either have no way to provide a mapping file as specified by the client's interface or because the attempted operation will fail (unable to find required metadata). In the case where the mapping file is used for simple mirroring, and the interface is simply the presence of a mapping file on the filesystem, the user may be unaware that the metadata they use is not coming from the source they expect. This should be solved by clear messaging around the source of the metadata during updating operations. Additionally, any targets downloaded under these assumptions still have the full guarantees of TUF metadata in general.

Because TAP4 does not address any security flaws, it is safe to allow old and new clients to coexist for a time.

TAP 5

Current Backwards Compatibility Section:

This specification is not backwards-compatible with clients that do not recognize TAP 5, because the changes to the root and snapshot metadata file has implications on how metadata files are downloaded. However, note that it should take very little effort to incorporate these changes with an existing implementation.

New Backwards Compatibility Section Proposal:

This TAP is backwards compatible with the previous version of TUF.

Unaware clients (that is, clients that don't understand TAP5's changes) will not understand the URLs field and will either fail or ignore it. If they ignore it, all metadata files will be fetched from the same repository as root.json, after which the target file will either exist in the metadata or fail to be found. If it succeeds, then the client may be putting more load on the repository than intended (by ignore mirrors listed in URLs) or it may be served a different version of the target (with the same name). If it fails, then the target was not found, and the URLs were used to serve alternate metadata, and a client failure is appropriate.

Aware clients will see metadata without URLs as existing in the same repository as the root file, which is the same as the pre-TAP5 case. Aware clients can ignore the root hash, if it exists in the snapshot file, so they will continue to work with repositories that publish data under the pre-TAP format.

Because TAP5 does not address any security flaws, it is safe to allow old and new clients to coexist for a time.

@trishankkarthik @JustinCappos (please loop in anyone else that may be interested, just didn't want to spam everyone)

[trishankkarthik]

Love it --- thanks for raising this issue! Let me read it carefully, and get back to you :)

[trishankkarthik]

Looks good to me! What do you think @JustinCappos ?

[JustinCappos]

I need to spend some time thinking deeply about whether there are any security issues lurking here, but I appreciate the suggestion and agree we should merge these sections in.

…

On Thu, Dec 8, 2016 at 11:55 AM, Trishank Karthik Kuppusamy < ***@***.***> wrote: Looks good to me! What do you think @JustinCappos <https://github.com/JustinCappos> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA0XD9kNa8RP_zQA-avulyRxLS13b73Cks5rGDaPgaJpZM4LH5ba> .

[vladimir-v-diaz]

@ecordell

This TAP can be made backwards compatible with the previous implementation of TUF by treating metadata with a name field the same as a metadata with a names field containing one entry.

I'd like to verify the wording here. The quoted text in bold is unclear to me. Do you mean to say that current implementations (compliant with the pre-TAP3 specification) can be made backwards compatible by doing this and that? Or are you proposing that backwards-compatibility be built into the TAP? In other words, that a TAP3-compliant implementation be forced to recognize previous formats. If this is indeed what you intend, then TAP3 must be updated to list the supported formats of a delegation (i.e., "names" and "names" are valid attributes).

I am implementing TAP3 and would like to verify the wording before taking a deep dive.

[ecordell]

@vladimir-v-diaz The way the TAP is currently written changes the specification to use the names key instead of the name key. So the proposal of supporting both is technically invalid against the current spec.

I mostly wrote it that way to signal that some small changes to the rest of the TAP are necessary to make the backwards compatibility section correct. I.e. if this were a PR instead of a discussion, I would've changed the part of TAP3 that defines the file format to allow name and names to be optional, and the wording in the backwards compatibility section would've been "This TAP is backwards compatible..."

Would it be helpful to issue PRs for these three sections to have TAP-specific discussions? I opened this because I think this requires careful consideration to ensure no security implications.

[awwad]

I think the modified text you (@ecordell) provide is much better, but I say that with the same understanding Vlad indicated: that a particular implementation can provide backward compatibility by doing what you've indicated. I'm still not clear on whether or not you're suggesting that TAP 3 should require that TUF support both the old and new formats (which differ not only in the contents of name/names, but must also in the presence of keyids and thresholds outside of name/names).

If the suggestion is that implementations can support pre- and post-TAP3 TUF formats by doing what you've mentioned, then I think that makes sense.

If, instead, the suggestion is that TAP 3 should propose that both schemes be acceptable, so that the TUF reference implementation supports both, then I'm less sure about that: it seems confusing.

[vladimir-v-diaz]

Note that the TAP 3 proposal (as it is currently exists on the repository) does more than just change "name" to "names". It also moves around the keyids, threshold, etc. attributes. Please compare the formats below:

current format:

"roles": [
    {
     "keyids": [
      "a0650f29dde63403cc4eec28a1c66f2262d6339434a01c63a881a48bedd9bca9"
     ],
     "name": "role1",
     "paths": [
      "/file3.txt"
     ],
     "terminating": false,
     "threshold": 1
    }
]

proposed format:

{
roles: [{          
          // Both roles must sign the same hashes and length of the following targets.
          "paths": ["baz/*.pkg"],
          "names": {
            ROLENAME-1: {
              "keyids": [KEYID-1],
              "threshold": THRESHOLD-1
            },
            ROLENAME-2: {
              "keyids": [KEYID-2],
              "threshold": THRESHOLD-2
            }
          },
          ...
        ]

[vladimir-v-diaz]

Would it be helpful to issue PRs for these three sections to have TAP-specific discussions? I opened this because I think this requires careful consideration to ensure no security implications.

We'd welcome pull requests. It can certainly help us reason about the proposed changes.