kfctl apply from pricate kfdef on private and enterprize github

[moficodes]

right now the only way we can deploy kubeflow with kfctl is if the manifest is coming from a public repository and the kfdef is either downloaded or is also a public url.

it would be useful if we added a way to download private github files using token and from enterprise github.

[moficodes]

/assign

[arun-vc]

This would be very useful since some component overlays need to be private

[soleares]

If you're trying to add a private github repo to your kfdef in order to specify custom versions of some applications this works now in master:

• Build kfdef from master. It needs this PR: Add back support for go-getter manifest repo uri's #247
  • git clone https://github.com/kubeflow/kfctl.git
  • cd kfctl
  • make install
• Append an entry to the repos list for your internal github repo. This assumes that you use SSH to access it.

  repos:
  - name: manifests
    uri: git::https://github.com/kubeflow/manifests/archive/v1.2.0.tar.gz
  - name: my-manifests
    uri: git::[email protected]:myorg/myrepo.git

• If you need to access a specific branch/tag/commit you can add a ref

  repos:
  - name: manifests
    uri: git::https://github.com/kubeflow/manifests/archive/v1.2.0.tar.gz
  - name: my-manifests
    uri: git::[email protected]:myorg/myrepo.git?ref=mybranch

[shalberd]

@pdmack @Tomcli I wonder if adding an environment variable SSL_CERT_DIR or SSL_CERT_FILE to the environment that embeds Kubeflow kfctl

@AlexanderEkdahl you seemed to be the one who initiated the fix for considering http and https proxies a while ago, thank you.

opendatahub-io/opendatahub-operator@16eba4a

Can you tell from your perspective whether SSL_CERT_DIR and SSL_CERT_FILE, one of the two, are enough to specifiy when making plain http requests with the golang client? That is, are the changes from package x509

https://go.dev/src/crypto/x509/root_unix.go

golang/go@e83bcd9

considered in the requests here

https://github.com/opendatahub-io/opendatahub-operator/blob/master/pkg/kfconfig/types.go#L530

using x509.SystemCertPool implicitly?

Before 2017, one needed to add it manually:

https://stackoverflow.com/questions/38822764/how-to-send-a-https-request-with-a-certificate-golang

keywords: caCertPool.AppendCertsFromPEM and rootCAs argument.

After 2017, it should be there already in x509.loadSystemRoots

If yes, that would accomplish the job of making it possible to trust enterprise PKI CAs. Kubeflow kfctl as well as opendatahub-operator are using net/http for downloading manifest files:

https://github.com/devgrok/kubeflow-kfctl/blob/master/pkg/kfconfig/types.go#L508

Does that imply that SSL_CERT_DIR and SSL_CERT_FILE envs are honored when doing chain of trust validation in go?