Candidate recommendation exit criteria

[samuelweiler]

The Director expects CR documents to explicitly state the criteria for getting to PR. We can set criteria of our choice, but the suggestion is: "In order for this specification to exit Candidate Recommendation there must be demonstrations of at least two interoperable implementations of every feature."

The general rule is that CRs "must document how adequate implementation experience will be demonstrated" and that in moving to PR we "must show adequate implementation experience except where an exception is approved by the Director." So we need to carefully set a standard that we expect to be able to meet.

We may identify features "at risk" or with different criteria than the bulk.

[samuelweiler]

Some of this is from https://www.w3.org/2018/Process-20180201/#candidate-rec

[samuelweiler]

here's an example: https://www.w3.org/TR/2014/CR-WebCryptoAPI-20141211/

For this specification to be advanced to Proposed Recommendation, there must be at least two independent, interoperable implementations of each feature.

Each cryptographic algorithm listed in the specification is currently an "at risk" feature. In particular, we are expecting a "browser profile" to be created of interoperable algorithms for Web browsers. To exit to Proposed Recommendation, each algorithm listed in the specification should have two interoperable implementations. Furthermore, there is an open dependency on the IRTF CFRG in terms of algorithm(s). These issues should be resolved by the interoperability testing necessary to exit CR.

[emlun]

Are the extensions included in those features? I.e., does every extension need to be implemented by at least two independent browsers and (where applicable) authenticators?

[samuelweiler]

I suggest we mark all extensions as "at risk" or at least set a lower bar for them.

[nadalin]

Since these are extensions and we are only testing input and output and all formats are same then I would say that extensions should be fine and not at risk Get Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Samuel Weiler <[email protected]> Sent: Wednesday, March 14, 2018 2:05:58 PM To: w3c/webauthn Cc: Subscribed Subject: Re: [w3c/webauthn] Candidate recommendation exit criteria (#837) I suggest we mark all extensions as "at risk" or at least set a lower bar for them. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebauthn%2Fissues%2F837%23issuecomment-373013667&data=04%7C01%7Ctonynad%40microsoft.com%7C44e19970616141219e3f08d589ac5542%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636566295615422808%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=yF8bA7PfeMGtJ2r81BPof2oblHAhjZOUUE4%2Ba6qjgS0%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHkeY6yeAv-muK7DtPClhLq5F3dd0J99ks5teRW1gaJpZM4SpM1q&data=04%7C01%7Ctonynad%40microsoft.com%7C44e19970616141219e3f08d589ac5542%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636566295615422808%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwifQ%3D%3D%7C-1&sdata=57o8HhGzDbuQx8PVe2l82W1sQ7C%2B08BkBEguiTtKbyg%3D&reserved=0>.

[equalsJeffH]

seems to me since this is "exit (CR) criteria" I'd attach it to the PR milestone....

[samuelweiler]

JeffH: We need this in the CR draft.

Here's another example: https://www.w3.org/TR/wake-lock/

[samuelweiler]

FYI, you might need to add this include file via a PR to:
https://github.com/tabatkins/bikeshed/tree/master/bikeshed/boilerplate/webauthn

[nadalin]

"For the Web Authentication specification to move to Proposed Recommendation we must show two independent, interoperable implementations of the Web Authentication API in browsers. We have already completed a portion of this work at two informal interops with implementations from three browsers (Chrome, Firefox, and Edge). These initial compliance tests have yielded relevant datasets. We also have a suite of initial compliance tests that have been committed to the W3C Web Platform Test repo and additional improvements to the tests will be required for the move to Proposed Recommendation."

[selfissued]

Per the call, I am to add a sentence to this description about the extensions framework and having multiple interoperable implementations of at least one extension to validate the extensions framework.

[nadalin]

yes

[selfissued]

All extensions are optional and all but AppID are at risk.

[samuelweiler]

"All extensions are optional. All except for appid are at risk."

[samuelweiler]

Consistent with https://lists.w3.org/Archives/Public/public-webauthn/2018Mar/0143.html, I propose the following text for the CR. Absent objection, I'll include this in the CR as published, and we'll need to backport it into the github source later:

For the Web Authentication specification to move to Proposed Recommendation we must show two independent, interoperable implementations of the Web Authentication API in browsers. We will also have multiple interoperable implementations of the AppID extension, validating the extensions framework. All other extensions are "at risk". If there are not multiple interoperable implementations, each may independently be removed or made informative at Proposed Recommendation.

We have had two informal interoperability tests with implementations in three browsers.

I observe that the "status of this doc" changes don't seem to be committed to either our repo nor the bikeshed repo, so my guess is that those changes are only in Angelo's local bikeshed environment. We need to get those committed at some point.

[selfissued]

@nadalin to talk with @AngeloKai about addressing this.

[AngeloKai]

@samuelweiler I just filed a PR for the CR include: speced/bikeshed#1285

[AngeloKai]

@samuelweiler can you please help draft the PR include? I started a PR here: speced/bikeshed#1286. But feel free to update it as you see fit. Same goes for @nadalin

[AngeloKai]

@samuelweiler The CR include was just merged. Please take a look.