sso-auth: issue with group validation

[weeco]

Describe the bug
I configured the buzzfeed SSO proxy to allow only specific groups inside of my google organization, but I can still login with an account which is not part of any group specified in the allowed groups.

I'll attach only some of the (relevant) redacted configurations before I paste all the configurations

Upstream config:

- service: kibana
  default:
    from: kibana.service.ops.company.com
    to: http://kibana-logging.logging:5601
  options:
    allowed_groups:
      - [email protected]
      - [email protected]

Logs when I am logging in with my not allowed user (which is only part of [email protected] but not part of group1 and group2):

{"error":"http: named cookie not present","level":"error","msg":"error authenticating user","remote_address":"x.x.x.x","service":"sso-proxy","time":"2018-11-27 16:17:25.11274"}
{"level":"info","msg":"starting OAuth flow","service":"sso-proxy","sign_in_url":{"Scheme":"https","Opaque":"","User":null,"Host":"sso-auth.service.ops.company.com","Path":"/sign_in","RawPath":"","ForceQuery":false,"RawQuery":"client_id=WTM4bkE3bWhPK0crMkp0QThTMWFwQUFkMWRrUkROcW0%3D\u0026redirect_uri=https%3A%2F%2Fkibana.service.ops.company.com%2Foauth2%2Fcallback\u0026response_type=code\u0026scope=\u0026sig=neUgpMO7aaAHxHHoj70RGot1e9glODgupdmBLM8ig3Y%3D\u0026state=9bsMNh4FKHWboLsCG_pwU9VmrUC5bgEqLukrwgM1QBVyN3qCPnMQpn0ltd17nIFw8O7CVj-eB6t8_6shM9keSTZlQyiquPbU5kaQwQaCC_3Jn0y7cETaei9b7Fnj8amIvMaLtC1VwNBQHrRroB90RuDRRVrvWLXa1m3o0qxHfwfpqwC5RzeokNYk_Jg9IwGMjr80PnfwAsDPs1wlbtiF7lQ%3D\u0026ts=1543335445","Fragment":""},"time":"2018-11-27 16:17:25.11274"}
{"action":"proxy","http_status":302,"level":"info","msg":"","remote_address":"x.x.x.x","request_duration":0.39268000000000003,"request_method":"GET","request_uri":"kibana.service.ops.company.com/ui/favicons/favicon-16x16.png","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"}
{"allowed_groups":null,"level":"info","msg":"validating groups","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"[email protected]"}
{"in_groups":[],"level":"info","msg":"authentication complete","remote_address":"x.x.x.x","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"[email protected]"}
{"action":"callback","http_status":302,"level":"info","msg":"","remote_address":"x.x.x.x","request_duration":2.855855,"request_method":"GET","request_uri":"kibana.service.ops.company.com/oauth2/callback?code=n4_dBI-clrawxIAbZC8JtUFshHisUEczKNxnCQE4lNENI9vwGucLU9UYIrcidACFIS-kaf2sOvWRjTi5sVBwagzIbW4EKn05IOyuyvnhzgpEh1baQMdt3hT6raowgzX_9EODkFRRDf5-6A6z-L50jH2UhZzmS_7pOwVzzWEr1d55c41nB0r75PSq7UCSnhLACTEdR8uz69jWfOjy3K2aguq1tu83x4apr8vF8LBqtnHOsR_lI4DKOzt9w8RN9xMA3XCEbSueJhRE-7e-sELaAndDOXPrs23od7rYo_pobJVrtVn1uEPGcwHIr2i5YzuW2EEu2ceqvVPAaf9pi6DvhWF4ge0VaeQMmOMZmNR11TX_YX4VpOzRcQ0Zhw_XvaukOFv-5YXuEAjuVJksfm890e0jav3Mo4VVDROPYhBvGrBJWpeNHKPns3RzLvhmfKI1g6Qy3UMSNCEh322cqCmh6LYAYqHycV_A5hFItRlYMFvXOPWV7k_VpI9_nSda5coc5bRLk_Br51g6TU1W4jm8DIna-tQVdQ3d6M5vlzArckYWKBW4A8rVSUbAtUA8_T89sigxg0i0Jx6MHES89S425jsPXzfGSCoATbSTtnq06KMX8uoGdscisRXe1npIb_6tWe9Mdnw%3D\u0026state=9bsMNh4FKHWboLsCG_pwU9VmrUC5bgEqLukrwgM1QBVyN3qCPnMQpn0ltd17nIFw8O7CVj-eB6t8_6shM9keSTZlQyiquPbU5kaQwQaCC_3Jn0y7cETaei9b7Fnj8amIvMaLtC1VwNBQHrRroB90RuDRRVrvWLXa1m3o0qxHfwfpqwC5RzeokNYk_Jg9IwGMjr80PnfwAsDPs1wlbtiF7lQ%3D","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"}
{"action":"proxy","http_status":200,"level":"info","msg":"","remote_address":"x.x.x.x","request_duration":3.2140099999999996,"request_method":"GET","request_uri":"kibana.service.ops.company.com/ui/favicons/favicon-16x16.png","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"[email protected]","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"}
{"action":"ping","http_status":200,"level":"info","msg":"","remote_address":"10.0.4.1:46784","request_duration":0.010053,"request_method":"GET","request_uri":"10.0.4.123:4180/ping","service":"sso-proxy","time":"2018-11-27 16:17:31.11274","user":"","user_agent":"kube-probe/1.11+"}

Expected behavior
Even if it is a missconfiguration on my side I'd expect definetely a more verbose log, which indicates during the login what groups are allowed, and what groups the logged user is attached to. Also I couldn't find any log lines which indicate that the service has successfully read all groups my google organization. I'd expect something like a list of read groups (or the number of successfully read groups using the google admin api credentials).

For instance the log line below should probably show the groups the user [email protected] is part of, right? If this is correct, this doesn't work apparently, and I haven't seen any error logs right after starting the service (e. g. "Couldn't read groups from google organization"), nor did I receive a message which says, yes it worked.

{"in_groups":[],"level":"info","msg":"authentication complete","remote_address":"x.x.x.x","service":"sso-proxy","time":"2018-11-27 16:17:25.11274","user":"[email protected]"}

Question:

How can I see if pulling the group information from my google organization did work or not?

Should the property in_groups during a login process show the organization groups which the user is part of?

[loganmeetsworld]

@weeco thanks for filing this issue. One thing that might be happening is that if the user has ever been a member of the group previously, it's possible with the way that we cache membership they are still a member of that group in the period before we flush the cache.

The point about logging is good. We could definitely do a better job with logging as I don't see that we log the groups anywhere and that could be helpful to catch this caching issue.

[weeco]

I restarted the pod multiple times to see if it would log anything on startup. The example user john.doe has never been part of the groups above. In my oppinion it looks like it hasn't been able to fetch the groups from the organization at all. I don't have any further logging to get any insights into this issue.

What could I do to resolve or further debug this issue right now? It's been a kinda disappointing process to migrate from the OAUth proxy to buzzfeed sso proxy until now, due to the lack of log messages

[loganmeetsworld]

@weeco thanks again for this issue. We're going to open a separate issue to address logging verbosity and specific places it could be improved and note this issue in it. #126

In terms of your problem, would you mind sharing what your accounts configuration file looks like?

[weeco]

What do you mean by "accounts configuration" file?

[loganmeetsworld]

Sorry for not being clear. I mean, following this step, the generated json file, redacted of course? Also, ensure you've followed these steps correctly. Ensure you the service account on the credentials page too.

I'm also ensuring we're handling the nil case here correctly, but cannot see any reason a user not in any groups would have universal access for a given email domain. Will keep looking.

[weeco]

I downloaded the service account JSON file from google and put it as secret into my kubernetes namespace. I can ensure it's working as the exact same secret is being used by the OAuth2 proxy.

I could imagine there is an issue with buzzfeed SSO proxy picking up the allowed groups config, but once again due to lacking logs I can not tell whether it's using the allowed groups configuration or no. Would be helpful to print the configuration settings with all hosts and it's allowed groups on startup (at least in a debug / verbose log level?

[loganmeetsworld]

We agree that logging could be more verbose in that, and other, area(s). It's something we hope to address in #126 either through our own work or a community contribution to add the logging. We're definitely going to discuss this in our issue triage at the end of the week.

One last question I have is - Is the name of the env var for the google account json file GOOGLE_SERVICE_ACCOUNT_JSON? This is the only way that it will be read in. Also the value of this env var should be the path to the file. One potential place for error could be that the path is misconfigured.

[weeco]

I will check that tomorrow, thank you for the support and the quick responses. Looking forward for the more verbose logging for sure :-).

[weeco]

Just checked what you asked for.

This is part of my deployment configuration:

    - name: GOOGLE_SERVICE_ACCOUNT_JSON
      value: /creds/sso-serviceaccount.json

I connected to one of the pods and I also ensured the json file is existent under this path. The structure of the JSON is a common service account (issued JSON file from Google Cloud). Do you still want me to post the redacted service account JSON contents?

[danbf]

@weeco have you also added in the GOOGLE_ADMIN_EMAIL var to the deployment config for the sso-auth container? it's the one that needs both the GOOGLE_SERVICE_ACCOUNT_JSON and GOOGLE_ADMIN_EMAIL in order to enable the use of group based access control.

refer you to #105 (comment) on where to add the config values

to flesh this out more, i’m positing that perhaps this is failing back to a non-group based access control since not everything needed for group based access control is present.

think you could provide a redacted version of the config vars you are passing to the sso-auth container?

[weeco]

Yes I did pass the environment variable to the deployment.

Sure I can put the redacted version of all kubernetes deployments online here on Monday

[vitalis]

I had the same issue what after configuring group validation using readme it didn't work.
I was able to resolve the issue after adding https://www.googleapis.com/auth/admin.directory.member.readonly scope to service account.
I think it's a good idea to update the README
@danbf

[bkix]

@vitalis I think the scope is missing the "group" and should be https://www.googleapis.com/auth/admin.directory.group.member.readonly

[nikogura]

It's your config. From the source, 'config' needs to be at the same level as 'to' and 'from'.

Try this:

- service: kibana
   default:
     from: kibana.service.ops.company.com
     to: http://kibana-logging.logging:5601
     options:
       allowed_groups:
         - [email protected]
         - [email protected]

If you don't have it at that level, the deserializer ignores it and it's the same as having specified no groups at all.

I had the same issue, and that did it for me.

[rajugupta15]

I am also hitting the same issue.

[namm2]

So I'm having the following errors when use allowed_groups options:

{"action":"redeem","http_status":200,"level":"info","msg":"","proxy_host":"example.com","remote_address":"1.2.3.4","request_duration":0.21089899999999998,"request_method":"POST","request_uri":"/redeem","service":"sso-authenticator","time":"2019-07-19 09:59:34.7199","user":"[email protected]","user_agent":"sso_proxy/HEAD"}
{"circuit_change_from":2,"circuit_change_to":1,"level":"info","msg":"circuit breaker trigger","service":"sso-authenticator","time":"2019-07-19 09:59:34.7199"}
{"circuit_change_from":1,"circuit_change_to":2,"level":"info","msg":"circuit breaker trigger","service":"sso-authenticator","time":"2019-07-19 09:59:35.7199"}
{"backoff_duration":557015243,"backoff_reset":"2019-07-19T09:59:35.592592582Z","level":"info","msg":"circuit breaker backoff set","service":"sso-authenticator","time":"2019-07-19 09:59:35.7199"}
{"error":"googleapi: Error 403: Not Authorized to access this resource/api, forbidden","level":"error","msg":"error retrieving groups","service":"sso-authenticator","time":"2019-07-19 09:59:35.7199"}
{"action":"profile","http_status":500,"level":"info","msg":"","proxy_host":"","remote_address":"1.2.3.4","request_duration":381.978062,"request_method":"GET","request_uri":"/profile?client_id=SjA5ck0zdlMxNi9KZGJHVjFVWlZzUmtFSH123456789%3D\u0026email=user.email%40company.com\u0026groups=admins%40company.com","service":"sso-authenticator","time":"2019-07-19 09:59:35.7199","user":"","user_agent":"sso_proxy/HEAD"}

With the configs look correct:

  upstreams:
    - service: servicename
      default:
        from: example.com
        to: http://servicename.namespace.svc.cluster.local:8080
        options:
          allowed_groups:
            - [email protected]

And I already double checked the Oauth scopes mentioned in the docs. Any ideas?

[svenmueller]

Any updates on this? We are also seeing the issue above.

[tl-adrian-bridgett]

I think I've found the cause. I don't know Go so lots of logging.Info was used :-) Submitted a PR:

#280

[Jusshersmith]

👋 I think this issue has become somewhat mixed up with different problems.

Regarding some of the last few messages referring to the below error:

{"error":"googleapi: Error 403: Not Authorized to access this resource/api, forbidden","level":"error","msg":"error retrieving groups","service":"sso-authenticator","time":"2019-07-19 09:59:35.7199"}

please take a look at #230 and mention there if you're still running into this error.

Regarding the initial error brought up, a lot has changed since this was opened or active so I'll close this off for now. However, for anyone that finds this error while searching, make sure you're not being caught out by #125 (comment) (good catch @nikogura!). Otherwise, please feel free to start a new issue if required!

Thank you!