Replace Dex by Oauth2-proxy?

[sayf-eddine-scality]

The main need/usage of dex is to have a resilient OIDC for Kubernetes API and have a fallback if additional authentication mechanisms fail.
We don't want an OIDC with database dependency, so we use the StaticPassword feature of Dex, but here lies the problem;
Dex does not keep track of the user authenticated through a static password (no session/cookie). So every time we move from one app to another we need to re-login.

The proposed feature is to use oauth2-proxy instead of Dex.

Oauth proxy provides:

• htpassword based authentication (similar to dex staticpassword but with session support)
• OIDC provider support
• seamless integration with Nginx ingress, (through annotation cf External authentication)
• passes user info through headers
• passes token id through cookies (needs a bit of logic to extract)

Issues to consider:

• oauth2-proxy supports one OIDC provider at a time (discussion in progress cf Enable multiple providers oauth2-proxy/oauth2-proxy#926)
• Does not provide authZ, (for upstreams that have no internal roles like Prometheus, we can put authZ logic in the ingress object through nginx configuration snippet)

Suggested flow

1. User access https://IP:8443/
2. Ingress controller send a request to oauth2-proxy and figure out user is not authenticated
3. [If user is not authenticated] User is redirected to oauth2-proxy to authenticate
4. Ingress controller receives authentication info (cookie, userinfo, headers)
5. [Optional] Ingress runs authZ logic and decides or not to proxy the upstream
6. Ingress controller passes auth info to the upstream
7. [Optional] Upstream runs authZ logic
8. User moves to another app
9. Ingress retrieve user info from oauth2-proxy (and seamlessly: GOTO step 5)

[gdemonet]

Looks good to me, for adding SSO to Ingresses.

I'm a bit curious however, what would this mean for K8s API authn (if we set it up to read from request headers) when using kubectl? Can we generate a functional kubeconfig? How would invalidation/refresh look like in this case?

[NicolasT]

I'm not sure I like having authn/authz logic in the ingress controller. It's an application concern... We shouldn't be too dependent on which ingress controller is being used.

[gdemonet]

Agreed, but some applications refuse to handle it (e.g. Prometheus), and request an intermediate proxy to take care of it. Would it be a problem to rely on some functionalities of our ingress controller for this? (I guess it would at least be a hindrance to port our "extra services" to other K8s distributions)

[NicolasT]

Can't oidc-proxy be configured to, well, proxy those services, with authn? Then the ingress could proxy to oidc-proxy, which then proxies to Prometheus etc. No need for any ingress-controller-specific features.

[sayf-eddine-scality]

We shouldn't be too dependent on which ingress controller is being used

Most ingresses support this pattern
cf Ambassador Auth service, Envoy external auth filter, AWS API gateway authorizer etc
But I agree it is not optimal to couple those two features

Can't oidc-proxy be configured to, well, proxy those services

by oidc-proxy did you mean oauth2-proxy itself or there is another component? and by not using the ingress itself, are we talking about a sidecar pattern?

[NicolasT]

I meant oauth2-proxy.