non-interactive "jsctl clusters connect" flow is broken

[amcginlay]

The standard behaviour is that the user dances around a browser connected to TLSPK to complete the auth. This is not appropriate in non-interactive sessions where we'd like the sequence to be automatable. To this aim the jsctl auth login --credentials ${creds_file} syntax was introduced. To obtain a creds file, the user needs to create a TLSPK service account from here (https://platform.jetstack.io/org/<ORG_NAME>/manage/service_accounts) which looks something like this:

{
  "user_id": "[email protected]",
  "user_secret": "S5wF+IP......o@7g1\\Ty/"
}

So far so good, but we (@inteon) believe the service accounts are insufficiently privileged because the following happens:

$ jsctl auth login --credentials creds.json
Login succeeded
$ jsctl config set organization gallant-wright
Your organization has been changed to gallant-wright
$ jsctl clusters connect kind_2302201811
failed to create service account: missing some roles in organization "gallant-wright": admin (403)

Perhaps, what we really want is the ability to download equivalent credentials file for an org "Member", rather than a "Service Account".

I can see no reason why this shouldn't be possible since this is no different to how, for example, an AWS user obtains their AccessKeyID/SecretAccessKey pair. NOTE that AWS provides two such pairs per user at any time to enable safe credential rotation which might be a requirement.

[amcginlay]

@achuchev believes the code tries to create a second service account and this could be the permissions blocker. If this is the case, then maybe the approach would be for the code to recognise that the passed-in creds belong to an existing service account and just recycle the one provided.

[amcginlay]

@achuchev on further inspection this would appear to be a bug on the platform/server side rather than the CLI. Would you agree? Should we move it? This issue is causing problems with a customer enablement task I promised to deliver. The non-interactive workaround is not pretty at all.

[amcginlay]

https://github.com/jetstack/preflight-platform/pull/5221/commits/f8986e901f7b27e1cea1767bba040f167daa1b5c fixes this problem. Closing.

[amcginlay]

I spoke too soon, the flow remains broken (see below) but the aforementioned fix still helps pure-API users.

$ jsctl auth login --credentials ~/Downloads/credentials.json
$ jsctl config set organization gallant-wright
$ jsctl clusters connect kind_2303071902
failed to create service account: missing some roles in organization "gallant-wright": admin (403)

[achuchev]

@amcginlay With the latest update, we have enabled service accounts to create pull image secrets. Your latest error indicates a service account cannot create another. Let us process this and get back to you.

[amcginlay]

@achuchev This explains the previous confusion and I was just about to message you. Since I'm bypassing jsctl in my own work and the platform issue has been successfully addressed, this is no longer a requirement from me. Unless anyone else raises this please de-prioritize/close. Many thanks for your hard work.

[amcginlay]

Also, just a heads up but this issue may be the reason I have a lot of (non-visible) auth0 errors littering the user space:

$ curl -X 'GET' \
  'https://platform.jetstack.io/api/v1/org/gallant-wright/users' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer eyJhbGc...GSgxw' | jq .
[
...,
  {
    "user_id": "auth0|638e6738ebd471c46bb50e09",
    "email": "auth0|638e6738ebd471c46bb50e09",
    "name": "<Error: Not Found>",
    "picture_url": "",
    "verified": false,
    "created_at": null,
    "last_login": null,
    "is_deleted": true,
    "roles": [
      "member"
    ]
  },
  {
    "user_id": "auth0|63933775f498fd42e2fd3b7b",
    "email": "auth0|63933775f498fd42e2fd3b7b",
    "name": "<Error: Not Found>",
    "picture_url": "",
    "verified": false,
    "created_at": null,
    "last_login": null,
    "is_deleted": true,
    "roles": [
      "member"
    ]
  },
  ...
]

[j-fuentes]

Hey, you both are correct that service accounts cannot create other service accounts. This was like that by design.

I that becomes a requirement I would like to stop and think about that carefully. That might require work that we will throw away soon in the VaaS platform if we want to implement it with fine-grain controls. As an alternative, we could let any service account create service accounts, but that is something we would have to study the implication of carefully.

[j-fuentes]

Also, just a heads up but this issue may be the reason I have a lot of (non-visible) auth0 errors littering the user space:

$ curl -X 'GET' \
  'https://platform.jetstack.io/api/v1/org/gallant-wright/users' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer eyJhbGc...GSgxw' | jq .
[
...,
  {
    "user_id": "auth0|638e6738ebd471c46bb50e09",
    "email": "auth0|638e6738ebd471c46bb50e09",
    "name": "<Error: Not Found>",
    "picture_url": "",
    "verified": false,
    "created_at": null,
    "last_login": null,
    "is_deleted": true,
    "roles": [
      "member"
    ]
  },
  {
    "user_id": "auth0|63933775f498fd42e2fd3b7b",
    "email": "auth0|63933775f498fd42e2fd3b7b",
    "name": "<Error: Not Found>",
    "picture_url": "",
    "verified": false,
    "created_at": null,
    "last_login": null,
    "is_deleted": true,
    "roles": [
      "member"
    ]
  },
  ...
]

I think this might be an indication that when we delete service account the relation between the service account and the organization is left behind and it is orphan. We would need to look into this, but I don't think it is related to service accounts not having permissions to create other service accounts.