Access to secure template repositories

[deboer-tim]

Currently Codewind only supports accessing template repos that are accessible via http. We should also support repos that are https (self-signed/any cert for now) and require authentication.

We should allow the user to optionally enter a user and password along with the repo URL, and use this when connecting to it. Priority should be on the Appsody and odo vNext repos, we do not need to support this for the Codewind style.

[rwalle61]

/assign

[rwalle61]

fyi GitHub is deprecating some authentication methods in October, which may affect our design https://developer.github.com/changes/2019-11-05-deprecated-passwords-and-authorizations-api/ .

However GitHub Enterprise seems to be unaffected for now:

This deprecation has not been applied to GitHub Enterprise offerings yet. Please check the latest Enterprise release notes to learn when this deprecation is initiated and which version of GitHub Enterprise Server will have password authentication removed.

[rwalle61]

To start with I can get cwctl to download templates from GHE repos via cwctl project create --url <url> --path <path> --username <username> --password <password>.

Beyond this it would be useful to flesh out the user experience, so we can design the implementation. For example it'd be good to know:

• what type of credentials we should support: username/password, Personal Access Token, OAuth token
• whether all the templates from a source will need the same credentials, or whether each template may need different credentials
• how the user will provide and store the credentials (through the IDE UI/ Env variables/ other config mechanism)

[rwalle61]

Scope for 0.12:

• support username/password credentials (not Personal Access Token, other OAuth token)
• assume all templates from a source share the same credentials
• cwctl project create --username --password can
  • clone a public GHE repo
  • clone a private GH repo
  • download a tar.gz release asset on a public GHE repo
  • download a tar.gz release asset on a private GH repo
  • see 2647.1 download template from secure repo codewind-installer#434
• cwctl templates repos add --username --password can add a GHE template repository (index.json) feat: 2647.2 add secure template repos to pfe #2747, feat: 2647.3 cwctl template repos add secure repos codewind-installer#443

Feedback from this:

• git credentials can expire - should we verify that they are still valid? when should we do so?
• when you list template repositories, should the secure repos have a secure: true field?

[rwalle61]

Next step is to support Personal Access Tokens as well as username/password

• PFE feat: POST /templates/repos accepts Personal Access Tokens #2762
• cwctl feat: 2647.4 add secure template repos via PAT codewind-installer#453

After that we need to support storing and reusing git credentials. We will need a solution that fits into the wider Kabanero/ICP4a story (fyi @cccanderson). We'll pause work on this until we hear more details

(One way we could do it is:

• e.g.cwctl will store credentials in the keychain
• whenever we want to download a template, we can use those credentials
• whenever we want to use PFE's template API, we can use those credentials
  )

[sishida]

Assigning myself to help with documentation work.

[thisguy-1]

/assign

[thisguy-1]

@tetchel @sishida
Per our meeting here are the steps with a draft of the copy to be reviewed and edited by @sishida. This is specifically for VScode but should be adaptable to Eclipse, IntelliJ, and Che:

Step 1
Action: User selects "add new source", which creates dropdown wizard.
Where: Template Source Manager

Step 2
Action: User enters the URL of new source
Where: First prompt
Content:

• title bar - "Add New Source" and 'i' icon linking to docs
• text input - sample url preview
• instruction - "Enter the URL to your template source's index file. (Press 'Enter' to confirm or 'Escape' to cancel.

Step 3
Action: If link requires additional information credentials, user pics method.
Where: Second dropdown prompt
Content:

• title bar - "Add New Source" and 'i' icon linking to docs
• text input - "Authentication is needed for <website.com...>. Select and authentication method:"
• Method Options:
  • Method 1 - Title: "Username and Password"; Description (below): "Enter credentials for <website.com>"
  • Method 2 - Title: "Access Token"; Description (below): "Personal Access Token used by services such as Github, Gitbucket, Gitlab, etc. or Service Account Tokens used by a stackhub."

Step 4
Action: Based on the method the user either enters token or enters username
Where: Third dropdown prompt
Content:

• title bar - "Add New Source" and 'i' icon linking to docs

• if Method 1 - text input - "username" (as text preview); instruction - "Enter the username used to login to <website.com> (Press 'Enter' to confirm or 'Escape' to cancel).

• if Method 2 - text input - "access token" (as text preview); instruction - "Enter the access token used to login to <website.com> (Press 'Enter' to confirm or 'Escape' to cancel).

Step 5 (only for method 1)
Action: if the user chose method 1, the user enter password

Where: fourth dropdown prompt

Content:

• title bar - "Add New Source" and 'i' icon linking to docs

• text input - "password" (as text preview)

• instruction - "Enter the username used to login to <website.com> (Press 'Enter' to confirm or 'Escape' to cancel)."

[tobespc]

The types of credential to support are

• Username / Password
• Personal Access Token
• Secure Access Token

These all need to be sent as the bearer token when making api requests. We can store the information in the keychain (much like we do for docker registry) through cwctl

[thisguy-1]

@tetchel
per our conversation, we should try to figure out at what point we can let the user to know if they entered the correct credentials as they enter information and avoid them having to start the wizard all over again if they entered something incorrectly.

[thisguy-1]

@tobespc

The types of credential to support are

• Username / Password
• Personal Access Token
• Secure Access Token

These all need to be sent as the bearer token when making api requests. We can store the information in the keychain (much like we do for docker registry) through cwctl

Are you saying that in the front end, these method should be separate? We decided to combine the Personal access token and the secure template token in one input (method 2, above).

[rwalle61]

Okay, so currently the user of cwctl must provide credentials when adding the secure template repository to PFE, and also when creating a project from one of those secure templates.

As per the design above, we want cwctl to let the user provide credentials just once (when adding the secure template repo), and not need to re-enter the credentials when creating a project from one of those secure templates.

I will make cwctl do this by storing the credentials in the keychain when the user calls cwctl templates repos add, and using those credentials when the user calls cwctl project create <templateURL>. This is similar to how cwctl stores and uses docker registry secrets

Future steps:

• handle cases when the user's credentials expire (the Secure Access Token will not timeout, but it, passwords and Personal Access Tokens can expire if changed by the user externally).
• handle cases when PFE restarts and tries to assemble the template list (it currently doesn't store the git credentials, so will be unable to get templates from secure template repos after a restart)

[sishida]

Hi, team, Just chiming in for ID that it would be great to get ID review for the messages that appear with this design. When you're ready with a PR that includes messages, please request an ID team member to look them over. Thank you!

[eharris369]

@sishida
Steps for Eclipse:
Step 1
Action: Right-click on a connection in the Codewind Explorer view and select Manage Template Sources
Where: Codewind Explorer view

Step 2
Action: Click the Add button to add a new template source
Where: Manage Template Sources dialog

Step 3
Action: Fill in the template source URL
Where: Add Template Source dialog

Step 4
Action: If the template source requires authentication, check Authentication required for this URL, choose the Authentication method and fill in the authentication details:

• Method Options:
  • Method 1 - Logon authentication (user fills in Username and Password)
  • Method 2 - Access token authentication (user fills in Access token)
    

Step 5
Action: If the user wishes to validate that the authentication works they can click Test Template Source
If the test is successful, the dialog message will display "The template source test was successful". If the test was not successful an error dialog will be shown and once dismissed the dialog message will display "The template source test was not successful"

Step 6
Action: Click Next. Fill in the Name and Description for the template source or if desired, change the values that were automatically filled in from the template source URL if available.

Step 7
Action: Click Finish

Step 8
Action: The template source shows in the Manage Template Sources dialog. Click OK to add the template source to the connection.

[tetchel]

It adds new steps to https://www.eclipse.org/codewind/workingwithtemplates.html#adding-your-template-sources-to-codewind

After step 2, you receive a prompt that says the source could not be accessed, and may require authentication.
Click Authenticate.
Select whether to authenticate using username and password, or an access token.
For GitHub, this doc details access tokens. Tokens used in Codewind must have the repo permission scope.
Enter your username and password, or your access token.
Codewind tests the new source to make sure it can access it now.
If the test succeeds, enter a name and description for the new source, if the source does not already have one.
The new source appears in the Manage Sources page.

[tetchel]

I want to link the remaining issue #3153 from this epic - we are having mixed results getting username and password authentication to work in GitHub and GHE.