Merge pull request #285 from getodk/issa/rework-path-tokens

rework path token authentication

docs/api.md

@@ -36,10 +36,14 @@ Here major and breaking changes to the API are listed by version.
 
 ODK Central v1.0 adds Public Links to the API, and makes one minor breaking change.
 
+**Added**:
+
+* The new [Public Link](/reference/forms-and-submissions/'-public-access-links) resource lets you create Public Access Links, granting anonymous browser-based access to submit to your Forms using Enketo.
+
 **Changed**:
 
 * The non-extended App User response no longer includes a `createdBy` numeric ID. To retrieve the creator of an App User, request the extended response.
-* We no longer reject the request if multiple authentication schemes are presented, and instead document the priority order of the different schemes [here](TODO).
+* We no longer reject the request if multiple authentication schemes are presented, and instead document the priority order of the different schemes [here](/reference/authentication).
 
 ### ODK Central v0.9
 
@@ -170,7 +174,7 @@ In practice, there are two types of Actors available in the system today:
 
 In a future version of the API, programmatic consumers will be more directly supported as their own Actor type, which can be granted limited permissions and can authenticate over **OAuth 2.0**.
 
-Next, you will find documentation on each of the three authentication methods described above. It is best not to present multiple credentials. If you do, the first _presented_ scheme out of Bearer, Basic, Cookie, then `/key` token will be used for the request.
+Next, you will find documentation on each of the three authentication methods described above. It is best not to present multiple credentials. If you do, the first _presented_ scheme out of `/key` token, Bearer, Basic, then Cookie will be used for the request. If the multiple schemes are sent at once, and the first matching scheme fails, the request will be immediately rejected.
 
 ## Session Authentication [/v1/sessions]
 
@@ -224,7 +228,7 @@ _(There is not really anything at `/v1/example`; this section only demonstrates
 
 #### Logging out [DELETE /v1/sessions/{token}]
 
-Logging out is not strictly necessary; all sessions expire 24 hours after they are created. But it can be a good idea, in case someone else manages to steal your token. To do so, issue a `DELETE` request to that token resource.
+Logging out is not strictly necessary for Web Users; all sessions expire 24 hours after they are created. But it can be a good idea, in case someone else manages to steal your token. It is also the way Public Link and App User access are revoked. To do so, issue a `DELETE` request to that token resource.
 
 + Parameters
     + token: `lSpAIeksRu1CNZs7!qjAot2T17dPzkrw9B4iTtpj7OoIJBmXvnHM8z8Ka4QPEjR7` (string, required) - The session bearer token, obtained at login time.
@@ -310,7 +314,9 @@ Today, there are two types of accounts: `Users`, which are the administrative ac
 
 Actors (and thus Users) may be granted rights via Roles. The `/roles` Roles API is open for all to access, which describes all defined roles on the server. Getting information for an individual role from that same API will reveal which verbs are associated with each role: some role might allow only `submission.create` and `submission.update`, for example.
 
-Right now, there are three predefined system roles: Administrator (`admin`), Project Manager (`manager`), and App User (`app-user`). Administrators are allowed to perform any action upon the server, while Project Managers are allowed to perform any action upon the projects they are assigned to manage. App Users are granted minimal rights: they can read Form data and create new Submissions on those Forms.
+Right now, there are four predefined system roles: Administrator (`admin`), Project Manager (`manager`), Data Collector (`formfill`), and App User (`app-user`). Administrators are allowed to perform any action upon the server, while Project Managers are allowed to perform any action upon the projects they are assigned to manage.
+
+Data Collectors can see all Forms in a Project and submit to them, but cannot see Submissions and cannot edit Form settings. Similarly, App Users are granted minimal rights: they can read Form data and create new Submissions on those Forms. While Data Collectors can perform these actions directly on the Central administration website by logging in, App Users can only do these things through Collect or a similar data collection client device.
 
 The Roles API alone does not, however, tell you which Actors have been assigned with Roles upon which system objects. For that, you will need to consult the various Assignments resources. There are two, one under the API root (`/v1/assignments`), which manages assignments to the entire system, and another nested under each Project (`/v1/projects/…/assignments`) which manage assignments to that Project.
 
@@ -1262,7 +1268,7 @@ Draft Forms allow you to test and fix issues with Forms before they are finalize
 
 You can create or replace the current Draft Form at any time by `POST`ing to the `/draft` subresource on the Form, and you can publish the current Draft by `POST`ing to `/draft/publish`.
 
-When a Draft Form is created, a Draft Token is also created for it, which can be found in Draft Form responses at `draftToken`. This token allows you to [submit test Submissions to the Draft Form](TODO) through clients like Collect. If the Draft is published or deleted, the token will be deactivated. But if you replace the Draft without first deleting it, the existing Draft Token will be carried forward, so that you do not have to reconfigure your device.
+When a Draft Form is created, a Draft Token is also created for it, which can be found in Draft Form responses at `draftToken`. This token allows you to [submit test Submissions to the Draft Form](/reference/forms-and-submissions/'-draft-submissions/creating-a-submission) through clients like Collect. If the Draft is published or deleted, the token will be deactivated. But if you replace the Draft without first deleting it, the existing Draft Token will be carried forward, so that you do not have to reconfigure your device.
 
 + Parameters
     + projectId: `1` (number, required) - The `id` of the Project this Form belongs to.
@@ -1673,7 +1679,7 @@ There are only one set of Roles, applicable to either scenario. There are not a
 
 + Parameters
     + projectId: `2` (number, required) - The numeric ID of the Project
-    + xmlFormId: `simple` (string, required) - The friendly name of this form. It is given by the `<title>` in the XForms XML definition.
+    + xmlFormId: `simple` (string, required) - The `xmlFormId` of the Form being referenced.
 
 ### Listing all Form Assignments [GET]
 
@@ -1721,7 +1727,7 @@ No `POST` body data is required, and if provided it will be ignored.
     + Attributes (Success)
 
 + Response 403 (application/json)
-    + Attributes (Error 403)
+
 
 ### Revoking a Form Role Assignment from an Actor [DELETE /v1/projects/{projectId}/forms/{xmlFormId}/assignments/{roleId}/{actorId}]
 
@@ -1737,6 +1743,68 @@ Given a `roleId`, which may be a numeric ID or a string role `system` name, and
 + Response 403 (application/json)
     + Attributes (Error 403)
 
+## › Public Access Links [/v1/projects/{projectId}/forms/{xmlFormId}/public-links]
+
+_(introduced: version 1.0)_
+
+Anybody in possession of a Public Access Link for a Form can use that link to submit data to that Form. Public Links are useful for collecting direct responses from a broad set of respondents, and can be revoked using the administration website or the API at any time.
+
+The API for Public Links is particularly useful, as it can be used to, for example, programmatically create and send individually customized and controlled links for direct distribution.
+
+To revoke the access of any Link, terminate its session `token` by issuing [`DELETE /sessions/:token`](/reference/authentication/session-authentication/logging-out).
+
++ Parameters
+    + projectId: `2` (number, required) - The numeric ID of the Project
+    + xmlFormId: `simple` (string, required) - The `xmlFormId` of the Form being referenced.
+
+### Listing all Links [GET]
+
+This will list every Public Access Link upon this Form.
+
+This endpoint supports retrieving extended metadata; provide a header `X-Extended-Metadata: true` to retrieve the Actor the Link was `createdBy`.
+
++ Response 200 (application/json)
+    This is the standard response, if Extended Metadata is not requested:
+
+    + Attributes (array[Public Link])
+
++ Response 200 (application/json; extended)
+    This is the Extended Metadata response, if requested via the appropriate header:
+
+    + Attributes (array[Extended Public Link])
+
+### Creating a Link [POST]
+
+To create a new Public Access Link to this Form, you must send at least a `displayName` for the resulting Actor. You may also provide `once: true` if you want to create a link that [can only be filled by each respondent once](https://blog.enketo.org/single-submission-surveys/). This setting is enforced by Enketo using local device tracking; the link is still distributable to multiple recipients, and the enforcement can be defeated by using multiple browsers or devices.
+
++ Request (application/json)
+    + Attributes
+        + displayName: `my public link` (string, required) - The name of the Link, for keeping track of. This name is displayed on the Central administration website but not to survey respondents.
+        + once: `false` (boolean, optional) - If set to `true`, an Enketo [single submission survey](https://blog.enketo.org/single-submission-surveys/) will be created instead of a standard one, limiting respondents to a single submission each.
+
+    + Body
+
+            { "displayName": "my public link", "once": false }
+
++ Response 200 (application/json)
+    + Attributes (Public Link)
+
++ Response 403 (application/json)
+    + Attributes (Error 403)
+
+### Deleting a Link [DELETE /v1/projects/{projectId}/forms/{xmlFormId}/public-links/{linkId}]
+
+You can fully delete a link by issuing `DELETE` to its resource. This will remove the Link from the system entirely. If instead you wish to revoke the Link's access to prevent future submission without removing its record entirely, you can issue [`DELETE /sessions/:token`](/reference/authentication/session-authentication/logging-out).
+
++ Parameters
+    + linkId: `42` (integer, required) - The numeric ID of the Link
+
++ Response 200 (application/json)
+    + Attributes (Success)
+
++ Response 403 (application/json)
+    + Attributes (Error 403)
+
 ## Submissions [/v1/projects/{projectId}/forms/{xmlFormId}/submissions]
 
 `Submission`s are available as a subresource under `Form`s. So, for instance, `/v1/projects/1/forms/myForm/submissions` refers only to the Submissions that have been submitted to the Form `myForm`.
@@ -3212,7 +3280,7 @@ These are in alphabetic order, with the exception that the `Extended` versions o
 + excelContentType: (string, optional) - If the Form was created by uploading an Excel file, this field contains the MIME type of that file.
 
 ## Draft Form (Form)
-+ draftToken: `lSpAIeksRu1CNZs7!qjAot2T17dPzkrw9B4iTtpj7OoIJBmXvnHM8z8Ka4QPEjR7` (string, required) - The test token to use to submit to this draft form. See [Draft Testing Endpoints](TODO).
++ draftToken: `lSpAIeksRu1CNZs7!qjAot2T17dPzkrw9B4iTtpj7OoIJBmXvnHM8z8Ka4QPEjR7` (string, required) - The test token to use to submit to this draft form. See [Draft Testing Endpoints](/reference/forms-and-submissions/'-draft-submissions).
 + enketoId: `abcdef` (string, optional) - If it exists, this is the survey ID of this draft Form on Enketo at `/enketo`. Authentication is not needed to access the draft form through Enketo.
 
 ## Extended Form Version (Form)
@@ -3251,6 +3319,12 @@ These are in alphabetic order, with the exception that the `Extended` versions o
 + forms: `7` (number, required) - The number of forms within this Project.
 + lastSubmission: `2018-04-18T03:04:51.695Z` (string, optional) - ISO date format. The timestamp of the most recent submission to any form in this project, if any.
 
+## Public Link (Actor)
++ once: `false` (boolean, optional) - If set to `true`, an Enketo [single submission survey](https://blog.enketo.org/single-submission-surveys/) will be created instead of a standard one, limiting respondents to a single submission each.
+
+## Extended Public Link (Public Link)
++ createdBy (Actor, required) - The full details about the `Actor` that created this `App User`.
+
 ## Key (object)
 + id: `1` (number, required) - The numerical ID of the Key.
 + public: `bcFeKDF3Sg8W91Uf5uxaIlM2uK0cUN9tBSGoASbC4LeIPqx65+6zmjbgUnIyiLzIjrx4CAaf9Y9LG7TAu6wKPqfbH6ZAkJTFSfjLNovbKhpOQcmO5VZGGay6yvXrX1TFW6C6RLITy74erxfUAStdtpP4nraCYqQYqn5zD4/1OmgweJt5vzGXW2ch7lrROEQhXB9lK+bjEeWx8TFW/+6ha/oRLnl6a2RBRL6mhwy3PoByNTKndB2MP4TygCJ/Ini4ivk74iSqVnoeuNJR/xUcU+kaIpZEIjxpAS2VECJU9fZvS5Gt84e5wl/t7bUKu+dlh/cUgHfk6+6bwzqGQYOe5A==` (string, required) - The base64-encoded public key, with PEM envelope removed.

lib/http/middleware.js

@@ -42,21 +42,15 @@ const versionParser = (request, response, next) => {
 const fieldKeyParser = (request, response, next) => {
   const match = /^\/key\/([^/]+)\//.exec(request.url);
 
-  const prefixKey = Option.of(match)
-    .map((m) => decodeURIComponent(m[1]))
-    .filter((k) => /^[a-z0-9!$]{64}$/i.test(k));
-  prefixKey.ifDefined(() => {
-    request.url = request.url.slice(match[0].length - 1);
-  });
+  const prefixKey = Option.of(match).map((m) => decodeURIComponent(m[1]));
+  prefixKey.ifDefined(() => { request.url = request.url.slice(match[0].length - 1); });
 
-  const queryKey = Option.of(request.query.st)
-    .map(decodeURIComponent)
-    .filter((k) => /^[a-z0-9!$]{64}$/i.test(k));
+  const queryKey = Option.of(request.query.st).map(decodeURIComponent);
   queryKey.ifDefined((token) => {
     delete request.query.st;
     // we modify the request url to ensure openRosa gives prefixed responses
     // per the requested token. we have to slice off the /v1.
-    request.originalUrl = `/v1/key/${token}${request.originalUrl.slice(3)}`;
+    request.originalUrl = `/v1/key/${token.replace(/\//g, '%2F')}${request.originalUrl.slice(3)}`;
   });
 
   request.fieldKey = Option.of(prefixKey.orElse(queryKey));

lib/http/preprocessors.js

@@ -36,8 +36,34 @@ const sessionHandler = ({ Session, User, Auth, crypto }, context) => {
 
   const authHeader = context.headers.authorization;
 
+  // If a field key is provided, we use it first and foremost. We used to go the
+  // other way around with this, but especially with Public Links it has become
+  // more sensible to resolve collisions by prioritizing field keys.
+  if (context.fieldKey.isDefined()) {
+    // Picks up field keys from the url.
+    // We always reject with 403 for field keys rather than 401 as we do with the
+    // other auth mechanisms. In an ideal world, we would do 401 here as well. But
+    // a lot of the ecosystem tools will prompt the user for credentials if you do
+    // this, even if you don't issue an auth challenge. So we 403 as a close-enough.
+    //
+    // In addition to rejecting with 403 if the token is invalid, we also reject if
+    // the token does not belong to a field key, as only field keys may be used in
+    // this manner. (TODO: we should not explain in-situ for security reasons, but we
+    // should explain /somewhere/.)
+
+    const key = context.fieldKey.get();
+    if (!/^[a-z0-9!$]{64}$/i.test(key)) return reject(Problem.user.insufficientRights());
+
+    return Session.getByBearerToken(key)
+      .then(getOrReject(Problem.user.insufficientRights()))
+      .then((session) => {
+        if ((session.actor.type !== 'field_key') && (session.actor.type !== 'public_link'))
+          return reject(Problem.user.insufficientRights());
+        return context.with({ auth: new Auth({ _session: session }) });
+      });
+
   // Standard Bearer token auth:
-  if (!isBlank(authHeader) && authHeader.startsWith('Bearer ')) {
+  } else if (!isBlank(authHeader) && authHeader.startsWith('Bearer ')) {
     // auth by the bearer token we found:
     return authBySessionToken(authHeader.slice(7), () => reject(Problem.user.authenticationFailed()));
 
@@ -67,23 +93,6 @@ const sessionHandler = ({ Session, User, Auth, crypto }, context) => {
           return reject(Problem.user.authenticationFailed());
         }));
 
-  } else if (context.fieldKey.isDefined()) {
-    // Picks up field keys from the url.
-    //
-    // If authentication is already provided via Bearer token, we reject with 401.
-    //
-    // In addition to rejecting with 401 if the token is invalid, we also reject if
-    // the token does not belong to a field key, as only field keys may be used in
-    // this manner. (TODO: we should not explain in-situ for security reasons, but we
-    // should explain /somewhere/.)
-    return Session.getByBearerToken(context.fieldKey.get())
-      .then(getOrReject(Problem.user.authenticationFailed()))
-      .then((session) => {
-        if ((session.actor.type !== 'field_key') && (session.actor.type !== 'public_link'))
-          return reject(Problem.user.authenticationFailed());
-        return context.with({ auth: new Auth({ _session: session }) });
-      });
-
   // Cookie Auth, which is more relaxed about not doing anything on failures.
   // but if the method is anything but GET we will need to check the CSRF token.
   } else if (context.headers.cookie != null) {

test/integration/api/app-users.js

@@ -198,14 +198,14 @@ describe('api: /projects/:id/app-users', () => {
 
 // Test the actual use of field keys.
 describe('api: /key/:key', () => {
-  it('should return 401 if an invalid key is provided', testService((service) =>
+  it('should return 403 if an invalid key is provided', testService((service) =>
     service.get('/v1/key/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/users/current')
-      .expect(401)));
+      .expect(403)));
 
   it('should reject non-field tokens', testService((service) =>
     service.post('/v1/sessions').send({ email: '[email protected]', password: 'alice' })
       .then(({ body }) => service.get(`/v1/key/${body.token}/users/current`)
-        .expect(401))));
+        .expect(403))));
 
   it('should passthrough to the appropriate route with successful auth', testService((service) =>
     service.login('alice', (asAlice) =>

test/integration/api/public-links.js

@@ -182,9 +182,9 @@ describe('api: /projects/:id/forms/:id/public-links', () => {
 
 // Test the actual use of public links.
 describe('api: /key/:key', () => {
-  it('should return 401 if an invalid key is provided', testService((service) =>
+  it('should return 403 if an invalid key is provided', testService((service) =>
     service.get('/v1/key/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/users/current')
-      .expect(401)));
+      .expect(403)));
 
   it('should allow cookie+public-link', testService((service) =>
     service.post('/v1/sessions')

test/integration/api/sessions.js

@@ -152,7 +152,7 @@ describe('api: /sessions', () => {
           .then((token) => asBob.delete('/v1/sessions/' + token)
             .expect(200)
             .then(() => service.get(`/v1/key/${token}/users/current`)
-              .expect(401))))));
+              .expect(403))))));
 
     it('should allow managers to delete project public link sessions', testService((service) =>
       service.login('bob', (asBob) =>
@@ -163,7 +163,7 @@ describe('api: /sessions', () => {
           .then((token) => asBob.delete('/v1/sessions/' + token)
             .expect(200)
             .then(() => service.get(`/v1/key/${token}/users/current`)
-              .expect(401))))));
+              .expect(403))))));
 
     it('should not allow app users to delete their own sessions', testService((service) =>
       service.login('bob', (asBob) =>

test/integration/api/submissions.js

@@ -13,7 +13,7 @@ describe('api: /submission', () => {
 
     it('should fail on authentication given broken credentials', testService((service) =>
       service.head('/v1/key/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/projects/1/submission')
-        .expect(401)));
+        .expect(403)));
   });
 
   describe('POST', () => {