wip: working on credentials/verify conformance

[brownoxford]

This PR is the beginning of the work to add conformance testing for the /credentials/verify endpoint.

Right now, the problem I'm trying to solve is that mutating the credential properties in the VC results in the signature being invalid, causing negative tests to fail for one of two reasons - either the input fails validation before the signature is checked, or the signature checks fail.

We need for there to be a single reason for test failure, and have to eliminate the bad signature by coming up with a way to sign invalid credentials. We definitely don't want verification to succeed if a 3rd party presents a malformed credential with a valid signature.

Currently, I am able to do that by using the @digitalbazaar/vc package internals, bypassing much of the input validation code. I'm running into trouble with id and type fields, for which the JSON-LD parser definitely does not like invalid values.

If anyone has any suggestions for how to proceed, I would love to hear them.

My current plan is to pre-generate VCs for the test suite using the generate-testdata.js script. The commented-out test cases in that file represent the items with which I am having trouble.

[brownoxford]

This is related to w3c-ccg/vc-api-verifier-test-suite#26

[aljones15]

BTW this looks like a good case for a map:

const invaludValues = new Map([
  ['issuer:array', ['did:example:123']],
 ['issuer:boolean', false]
])

then you get an iterable by default

Also you might want to change the VC context here to allow for the invalid values

  "issuer": {"@id": "cred:issuer", "@type": "@id"},

   "issuer": {"@id": "cred:issuer", "@type": "xsd:boolean"}

Basically create an invalid VC context to allow for invalid values.

[brownoxford]

@aljones15

When I modify issuer in the context in this way, and try to set issuer: {"id": false} in the credential, I still get the same error I get without modifying the context:

JsonLdError [jsonld.SyntaxError]: Invalid JSON-LD syntax; "@id" value must a string.

Note that setting issuer: false does not cause any errors, nor does issuer: {"id": "did:web:example.com"}

[brownoxford]

This appears to be working:

        context['@context'].VerifiableCredential['@context'].issuer = {
          '@id': 'cred:issuer',
          '@type': '@id',
          '@context': {
            id: 'xsd:boolean',
          },
        };

[brownoxford]

Mutating any type property or the @context to anything invalid continues to prove difficult.

[dlongley]

Mutating any type property or the @context to anything invalid continues to prove difficult.

I just have to say ... that's great! :P ... the above is a sign of a good implementation / reasonably safe APIs. We want it to be hard to create invalid stuff -- since the only use case is a negative test suite.

[TallTed]

@dlongley -- Your turn to have an un-fenced @context in the quoted content that starts #400 (comment). Please edit to add code fences.

[brownoxford]

I'm moving this out of draft and making it ready for review. The current implementation provides a script that will regenerate a number of malformed (but validly signed) verifiable credentials for use with various bad request tests of the /credentials/verify endpoint.

VC generation is idempotent, and generated VCs are injected into the Postman testing suite to allow for easy regeneration of test request bodies. Note that updated postman JSON is output to stdout and must be captured and saved to the conformance_suite.postman_collection.json conformance suite JSON file.

Most mutations are able to be tested, but a number of mutations that change the @context or any type properties to bad values are not currently possible, largely because the RDF canonicalization algorithms simply will not work, and no signature can be generated.

[mkhraisha]

There are a number of commented-out lines, which will make it difficult going forward for anyone to delete/modify because they will be scared to remove those lines. I think a better path forward is to remove those comments.
Unless you believe a forthcoming PR that is reasonably soon will tackle those test issues.

[mkhraisha]

nit: This won't resolve (obviously), do we want to maybe use an issuer that would resolve as a valid example?

[brownoxford]

hmm. The goal of this request is to fail in the right way. If we think that an implementation might fail because the issuer cannot be resolved, then I would say yes, we should change this to something that will resolve. Suggestions welcome.

[brownoxford]

Agree 100% - we should use did:key for the issuer.

[brownoxford]

Issue #404 has been created to track this change.