[Enhancement] Add support for SAML ACS url with new prefix of _plugin

[RyanL1997]

Description

Enhance the router for saml endpoint to support both legacy (_opendistro) and new prefix (_plugin).
This PR is registering 2 new endpoints with _plugin prefix for handling saml acs + saml idp initiated acs. Compare to the legacy endpoints, the logic is the same, and it has been extracted out for better usage.

• Relate [BUG] auth/http/saml/Saml2SettingsProvider.java still using _opendistro/ instead of _plugins/ security#1941

Mitigation Concern

After merging this, we can kick off the replacement of the endpoint generation in backend codebase (source code). If we merge opensearch-project/security#3246, the new acs endpoint with _plugin prefix will be applied, however, the user with the legacy set up will need to add these endpoints into the allowlist for their opensearch_dashboards.yml. Here is a comparison:

Legacy: 
...
server.xsrf.allowlist: [ 
                        "/_opendistro/_security/saml/acs", 
                        "/_opendistro/_security/saml/acs/idpinitiated", 
                        "/_opendistro/_security/saml/logout"
                        ]
...
New:
...
server.xsrf.allowlist: [
                        "/_plugin/_security/saml/acs", 
                        "/_plugin/_security/saml/acs/idpinitiated", 
                        "/_opendistro/_security/saml/logout"
                        ]
...

Category

[Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation]
Enhancement

Issues Resolved

• Relate [BUG] auth/http/saml/Saml2SettingsProvider.java still using _opendistro/ instead of _plugins/ security#1941

Check List

• New functionality includes testing
• New functionality has been documented
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #1565 (c4e91c4) into main (f655ccf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1565   +/-   ##
=======================================
  Coverage   66.18%   66.18%           
=======================================
  Files          93       93           
  Lines        2339     2339           
  Branches      312      312           
=======================================
  Hits         1548     1548           
  Misses        722      722           
  Partials       69       69           

[cwperks]

@RyanL1997 is it possible to make it so that a user does not have to migrate config?

i.e.

server.xsrf.allowlist: ["/_opendistro/_security/saml/acs", "/_opendistro/_security/saml/logout"]

is equivalent to

server.xsrf.allowlist: ["/(_opendistro|_plugins)/_security/saml/acs", "/(_opendistro|_plugins)/_security/saml/logout"]

[RyanL1997]

Update for the mitigation concern:

How SAML Works:

graph TD;
    A[User] -->|Access Dashboard| B[OpenSearch Dashboard]
    B -->|Redirect to IdP| C[Identity Provider]
    C -->|Authenticate User| C
    C -->|Generate SAML Assertion| D[ACS URL in OpenSearch Dashboard]
    D -->|Validate SAML Assertion| E[OpenSearch Backend - security plugin]
    E -->|Validate/Map User| B
    B -->|Authenticated Session| A

Loading

1. User Accesses OpenSearch Dashboard:

   • When a user attempts to access the OpenSearch Dashboard, they are usually not authenticated.

2. Redirect to Identity Provider (IdP):

   • OpenSearch Dashboard redirects the user to the Identity Provider for authentication. This is often a separate service specialized in identity verification, like Okta, OneLogin, or Microsoft's ADFS.

3. Authentication by IdP:

   • The Identity Provider challenges the user to provide credentials (usually a username and password, but possibly multi-factor authentication). Once authenticated...

4. SAML Assertion Creation:

   • The IdP creates a SAML Assertion, which is a package of information that includes the user's identity and possibly other information (like roles or groups they belong to).

5. Sending to ACS URL:

   • This is where the ACS URL comes into play. The IdP sends the SAML Assertion back to the ACS URL specified in the SAML settings of the OpenSearch Dashboard (This is the one we changed in [WIP] Change the saml acs endpoint to support _plugin  security#3246). The ACS URL is essentially an endpoint on the OpenSearch Dashboard server that's configured to receive and parse these SAML Assertions.

6. Assertion Validation:

   • OpenSearch Dashboard, more precisely the OpenSearch backend (security plugin), validates the received SAML Assertion to make sure it's legitimate. This is done by checking various things like the signature, issuer, and the assertions within the SAML response.

7. User Validation/Mapping:

   • Once the SAML Assertion is validated, OpenSearch Backend might further validate or map the user to internal roles or permissions based on the SAML Assertion.

8. Authenticated Session:

   • Finally, the user is considered authenticated and is allowed to use OpenSearch Dashboard.

Setting Up ACS URL
When setting up SAML integration, one of the crucial steps is to configure the ACS URL correctly. This URL needs to be registered in the Identity Provider. When the IdP sends the SAML Assertion back, it sends it to this URL. Any mistake in setting up this URL usually results in failed authentication.

Problem

According to the above description of setting up ACS URL, the problem we are facing is that the library we are using - SAML Java Toolkit, even with the latest version (2.9.0) is not supporting setup multiple ACS urls. Circling back to our fix, if we apply this change - opensearch-project/security#3246 into the security backend, it will only tell the IDP to use the url with the _plugin prefix, so that, it doesn't matter if we choose to keep the endpoint registries or not in the frontend, the ACS url with the old prefix is never gonna be used, since the IDP we are using only takes "1" url as the ACS endpoint which is the new prefix one.

Solution

1. Option 1: Switch library - switch to a library that support multiple ACS config

• pros:
  • support both legacy and new endpoint, without the deprecation of the legacy one
• cons:
  • security: Each additional endpoint potentially increases the attack surface.
  • compatibility concern for current SAML flow
  • may introduce new configuration for let user to configure

2. Option 2: As @cwperks mentioned above, we deprecate the legacy ACS url but with a work around on the configuration for legacy users. (Recommended)

• pros:
  • User doesn't need to change/configure anything
  • Nothing need to be changed for 3.0.0, since we have switch to new ACS url
• cons:
  • may cause some confusions in the configuration

[peternied]

@RyanL1997 When you've got some cycles please create a PR to add ^^^ to https://github.com/opensearch-project/security/blob/main/ARCHITECTURE.md

[RyanL1997]

@RyanL1997 When you've got some cycles please create a PR to add ^^^ to https://github.com/opensearch-project/security/blob/main/ARCHITECTURE.md

Nice call out. I will do that.

[RyanL1997]

Option 2- Configuration approach works, and if we choose to choose this approach, we need to merge the PR in following orders:
Security frontend changes to support new ACS url: #1565
Security backend changes to register ACS url with new prefix: opensearch-project/security#3246
Dashboard core change to modify the xsrf.allowList if the user input the legacy url : opensearch-project/OpenSearch-Dashboards@main...RyanL1997:OpenSearch-Dashboards:xsrfallowlist-update

[RyanL1997]

Closing this according to opensearch-project/security#3271