feat: third party custom providers

* feat: third party custom providers

- New configuration option `third_party.custom_providers`. `custom_providers`
  is a map of arbitrarily chosen keys to a `CustomThirdPartyProvider` - this is
  implemented as a new type differing from the existing configuration type
  `ThirdPartyProvider` used for built-in providers because they have different
  configuration requirements.

- Both `ThirdPartyProvider` and `CustomThirdPartyProvider` types get a non-
  configurable, automatically populated `Name` (during the config's `PostProcess`)
  that sort of serves as an identifier/slug for the provider in order to
  distinguish provider types at runtime.
    - A `CustomThirdPartyProvider`s `Name` is automatically prefixed during
      `PostProcess` with a "custom_" prefix to ensure that providers can be
      distinguished at runtime.
    - A (built-in) `ThirdPartyProvider`s `Name` is "hard-coded" through the
      `DefaultConfig`.

- Built-in OAuth/OIDC provider implementations are currently instantiated
  on-demand instead of once at appliation startup (i.e. unlike SAML
  providers) - i.e. when a user requests auth/authz with a third party
  provider, only then a provider is instantiated and created via factory
  function (`thirdparty.GetProvider`).  Custom providers follow this
  pattern, hence the factory function had to be adjusted to take into account
  providers with the aforementioned "custom_" prefix (i.e.: if it is a
  "custom_" provider, instantiate a `customProvider` implementation).

- The `customProvider` implementation uses the `go-oidc` library. Instances
  of providers of the type this library offers can be instantiated by passing
  in an `issuer` URL. Such an instantiation automatically attempts to retrieve
  an OIDC discovery document from a `.well-known` endpoint. This also performs
  an issuer validation. Providers configured to not use OIDC discovery (i.e.
  `use_discovery` in the `CustomThirdPartyProvider` is `false` or omitted) do
  not do this issuer check.

- The `customProvider` implementation is further based on the assumption that
  provider user data is only extracted from a userinfo endpoint response, i.e.
  in case of an OIDC provider, the implementation does not make use of the ID
  token - no validation is performed on the ID token.

- The `customProvider` implementation requires configuring a list of `scopes`:
  because the custom providers allow configuring both OAuth as well as OIDC
  providers, we cannot simply set a default set of scopes, e.g. `openid`, which
  is a required claim for OIDC - some providers return errors on unknown claims
  so setting this would make the third party auth process prone to errors.

- The `customProvider` implementation allows for a simple mapping of claims
  contained in the userinfo response from the provider to "known" standard OIDC
  conformant claims at the Hanko backend (defined in the `thirdparty.Claims`
  struct) through an `attribute_mapping` configuration option. The mapping is a
  simple one-to-one mapping, i.e. no complex mapping instructions are possible,
  e.g. mapping concatenations of multiple claims in the provider data source or
  similar. Any other non-standard claims returned by the provider are placed in
  a `custom_claims` attribute. Except for the user ID (`sub`), `email` and
  `email_verified` claims the third party functionality currently does not allow
  accessing this user data but there's a good chance this will change in the future,
  so I tried to make sure that any info retrieved from the provider is somehow
  preserved (it is persisted in the `data` column for an `identity` btw. and updated
  on every login with the provider).
    - I also noticed that the `thirdparty.Claims` were missing the `address` claim,
      so I added that for completeness' sake.

- The changes also fix a "bug" in the account `linking` logic whereby third party
  connections were established by simply assuming that the email retrieved from the
  provider was verified. So, even if the email address at the provider was not
  verified (or the provider simply did/does not provide info about the verification
  status) an account was created and/or linked and the flow API capabilities of
  automatically triggering a passcode if the backend was configured to require email
  verification would not take effect. This was a wrong assumption and the verification
  status is now based on the actual value retrieved from the provider.
- In case of a triggered passcode, the changes also modify the token exchange
   action to prevent showing a `back` button/link, since it does not make sense to
   go `back` to anything right after the token exchange - there is nothing to go
   "back" to.

backend/README.md

@@ -17,7 +17,10 @@ easily integrated into any web app with as little as two lines of code.
   - [Cross-domain communication](#cross-domain-communication)
   - [Audit logs](#audit-logs)
   - [Rate Limiting](#rate-limiting)
-  - [Social logins](#social-logins)
+  - [Social connections](#social-connections)
+    - [Built-in providers](#built-in-providers)
+    - [Custom OAuth/OIDC providers](#custom-oauthoidc-providers)
+    - [Account linking](#account-linking)
   - [User import](#user-import)
   - [Webhooks](#webhooks)
 - [API specification](#api-specification)
@@ -432,13 +435,22 @@ It uses a combination of user-id/IP to mitigate DoS attacks on user accounts. Yo
 In production systems, you may want to hide the
 Hanko service behind a proxy or gateway (e.g. Kong, Traefik) to provide additional network-based rate limiting.
 
-### Social Logins
+### Social connections
 
 Hanko supports OAuth-based ([authorization code flow](https://www.rfc-editor.org/rfc/rfc6749#section-1.3.1)) third
-party provider logins. See the `third_party` option in
-the [configuration reference](https://github.com/teamhanko/hanko/wiki/hanko-properties-third_party) on how to
-configure them. All provider configurations require provider credentials. See the guides in the official
-documentation for instructions on how to obtain these:
+party provider logins. The `third_party` configuration
+[option](https://github.com/teamhanko/hanko/wiki/config-properties-third_party) contains all relevant configuration.
+This includes options for setting up redirect URLs (in case of success or error on authentication with a provider) that
+apply to both [built-in](#built-in-providers) and
+[custom](#custom-oauthoidc-providers) providers.
+
+
+#### Built-in providers
+
+Built-in providers can be configured through the `third_party.providers` configuration [option](https://github.com/teamhanko/hanko/wiki/config-properties-third_party).
+They must be explicitly `enabled` (i.e. providers are disabled default).
+All provider configurations require provider credentials in the form of a client ID (`client_id`)
+and a client secret (`secret`). See the guides in the official documentation for instructions on how to obtain these:
 
 - [Apple](https://docs.hanko.io/guides/authentication-methods/oauth/apple)
 - [Discord](https://docs.hanko.io/guides/authentication-methods/oauth/discord)
@@ -447,9 +459,21 @@ documentation for instructions on how to obtain these:
 - [LinkedIn](https://docs.hanko.io/guides/authentication-methods/oauth/linkedin)
 - [Microsoft](https://docs.hanko.io/guides/authentication-methods/oauth/microsoft)
 
+#### Custom OAuth/OIDC providers
+
+Custom providers can be configured through the `third_party.custom_providers` configuration
+[option](https://github.com/teamhanko/hanko/wiki/config-properties-third_party-properties-custom_providers).
+Like built-in providers they must be explicitly `enabled` and require a `client_id` and `secret`, which must
+be obtained from the respective provider.
+Custom providers can use either OAuth or OIDC. OIDC providers can be configured to use
+[OIDC Discovery](https://openid.net/specs/openid-connect-discovery-1_0.html) by setting the `use_discovery`
+option to `true`. An `issuer` must be configured too in that case. Otherwise both OAuth and OIDC providers
+can manually define required endpoints (`authorization_endpoint`, `token_endpoint`, `userinfo_endpoint`).
+`scopes` must be explicitly defined (with `openid` being the minimum requirement in case of OIDC providers).
+
 #### Account linking
 
-The `allow_linking` configuration option for providers determines whether automatic account linking for this provider
+The `allow_linking` configuration option for built-in and custom providers determines whether automatic account linking for this provider
 is activated. Note that account linking is based on e-mail addresses and OAuth providers may allow account holders to
 use unverified e-mail addresses or may not provide any information at all about the verification status of e-mail
 addresses. This poses a security risk and potentially allows bad actors to hijack existing Hanko

backend/config/config_default.go

@@ -121,26 +121,32 @@ func DefaultConfig() *Config {
 				Apple: ThirdPartyProvider{
 					DisplayName:  "Apple",
 					AllowLinking: true,
+					Name:         "apple",
 				},
 				Discord: ThirdPartyProvider{
 					DisplayName:  "Discord",
 					AllowLinking: true,
+					Name:         "discord",
 				},
 				LinkedIn: ThirdPartyProvider{
 					DisplayName:  "LinkedIn",
 					AllowLinking: true,
+					Name:         "linkedin",
 				},
 				Microsoft: ThirdPartyProvider{
 					DisplayName:  "Microsoft",
 					AllowLinking: true,
+					Name:         "microsoft",
 				},
 				GitHub: ThirdPartyProvider{
 					DisplayName:  "GitHub",
 					AllowLinking: true,
+					Name:         "github",
 				},
 				Google: ThirdPartyProvider{
 					DisplayName:  "Google",
 					AllowLinking: true,
+					Name:         "google",
 				},
 			},
 		},

backend/config/config_third_party.go

@@ -13,6 +13,8 @@ import (
 type ThirdParty struct {
 	// `providers` contains the configurations for the available OAuth/OIDC identity providers.
 	Providers ThirdPartyProviders `yaml:"providers" json:"providers,omitempty" koanf:"providers" jsonschema:"title=providers,uniqueItems=true"`
+	// `custom_providers contains the configurations for custom OAuth/OIDC identity providers.
+	CustomProviders CustomThirdPartyProviders `yaml:"custom_providers" json:"custom_providers,omitempty" koanf:"custom_providers" jsonschema:"title=custom_providers"`
 	// `redirect_url` is the URL the third party provider redirects to with an authorization code. Must consist of the base URL
 	// of your running Hanko backend instance and the `callback` endpoint of the API,
 	// i.e. `{YOUR_BACKEND_INSTANCE}/thirdparty/callback.`
@@ -54,7 +56,10 @@ type ThirdParty struct {
 }
 
 func (t *ThirdParty) Validate() error {
-	if t.Providers.HasEnabled() {
+	hasEnabledProviders := t.Providers.HasEnabled()
+	hasEnabledCustomProviders := t.CustomProviders.HasEnabled()
+
+	if hasEnabledProviders || hasEnabledCustomProviders {
 		if t.RedirectURL == "" {
 			return errors.New("redirect_url must be set")
 		}
@@ -78,9 +83,18 @@ func (t *ThirdParty) Validate() error {
 		}
 	}
 
-	err := t.Providers.Validate()
-	if err != nil {
-		return fmt.Errorf("failed to validate third party providers: %w", err)
+	if hasEnabledProviders {
+		err := t.Providers.Validate()
+		if err != nil {
+			return fmt.Errorf("failed to validate third party providers: %w", err)
+		}
+	}
+
+	if hasEnabledCustomProviders {
+		err := t.CustomProviders.Validate()
+		if err != nil {
+			return fmt.Errorf("failed to validate custom third party providers: %w", err)
+		}
 	}
 
 	return nil
@@ -97,9 +111,192 @@ func (t *ThirdParty) PostProcess() error {
 		t.AllowedRedirectURLMap[redirectUrl] = g
 	}
 
+	if t.CustomProviders != nil {
+		providers := make(map[string]CustomThirdPartyProvider)
+		for key, provider := range t.CustomProviders {
+			// add prefix per default to ensure built-in and custom providers can be distinguished
+			keyLower := strings.ToLower(key)
+			provider.Name = "custom_" + keyLower
+			providers[keyLower] = provider
+		}
+		t.CustomProviders = providers
+	}
+
 	return nil
 }
 
+type CustomThirdPartyProviders map[string]CustomThirdPartyProvider
+
+func (p *CustomThirdPartyProviders) GetEnabled() []CustomThirdPartyProvider {
+	var enabledProviders []CustomThirdPartyProvider
+	for _, provider := range *p {
+		if provider.Enabled {
+			enabledProviders = append(enabledProviders, provider)
+		}
+	}
+
+	return enabledProviders
+}
+
+func (p *CustomThirdPartyProviders) HasEnabled() bool {
+	for _, provider := range *p {
+		if provider.Enabled {
+			return true
+		}
+	}
+
+	return false
+}
+
+func (p *CustomThirdPartyProviders) Validate() error {
+	for _, v := range p.GetEnabled() {
+		err := v.Validate()
+		if err != nil {
+			return fmt.Errorf(
+				"failed to validate third party provider %s: %w",
+				strings.TrimPrefix(v.Name, "custom_"),
+				err,
+			)
+		}
+	}
+	return nil
+}
+
+type CustomThirdPartyProvider struct {
+	// `allow_linking` indicates whether existing accounts can be automatically linked with this provider.
+	//
+	// Linking is based on matching one of the email addresses of an existing user account with the (primary)
+	// email address of the third party provider account.
+	AllowLinking bool `yaml:"allow_linking" json:"allow_linking,omitempty" koanf:"allow_linking" jsonschema:"default=false"`
+	// `attribute_mapping` defines a map that associates a set of known standard OIDC conformant end-user claims
+	// (the key of a map entry) at the Hanko backend to claims retrieved from a third party provider (the value of the
+	// map entry). This is primarily necessary if a non-OIDC provider is configured/used in which case it is probable
+	// that user data returned from the userinfo endpoint does not already conform to OIDC standard claims.
+	//
+	// Example: You configure an OAuth Provider (i.e. non-OIDC) and the provider's configured userinfo endpoint returns
+	// an end-user's user ID at the provider not under a `sub` key in its JSON response but rather under a `user_id`
+	// key. You would then configure an attribute mapping as follows:
+	//
+	// ```yaml
+	//attribute_mapping:
+	//  sub: user_id
+	// ```
+	//
+	// See https://openid.net/specs/openid-connect-core-1_0.html#StandardClaims for a list of known standard claims
+	// that provider claims can be mapped into. Any other claims received from a provider are not discarded but are
+	// retained internally in a `custom_claims` claim.
+	//
+	// Mappings are one-to-one mappings, complex mappings (e.g. mapping concatenations of two claims) are not possible.
+	AttributeMapping map[string]string `yaml:"attribute_mapping" json:"attribute_mapping,omitempty" koanf:"attribute_mapping"`
+	// URL of the provider's authorization endpoint where the end-user is redirected to authenticate and grant consent for
+	// an application to access their resources.
+	//
+	// Required if `use_discovery` is false or omitted.
+	AuthorizationEndpoint string `yaml:"authorization_endpoint" json:"authorization_endpoint,omitempty" koanf:"authorization_endpoint"`
+	// `name` is a unique identifier for the provider, derived from the key in the `custom_providers` map, by 
+	// concatenating the prefix "custom_". This allows distinguishing between built-in and custom providers at runtime.
+	Name string `jsonschema:"-"`
+	// `issuer` is the provider's issuer identifier. It should be a URL that uses the "https"
+	//	scheme and has no query or fragment components.
+	//
+	// Required if `use_discovery` is true.
+	Issuer string `yaml:"issuer" json:"issuer,omitempty" koanf:"issuer"`
+	// `client_id` is the ID of the OAuth/OIDC client. Must be obtained from the provider.
+	//
+	// Required if the provider is `enabled`.
+	ClientID string `yaml:"client_id" json:"client_id" koanf:"client_id" split_words:"true"`
+	// `display_name` is the name of the provider that is intended to be shown to an end-user.
+	DisplayName string `yaml:"display_name" json:"display_name" koanf:"display_name" jsonschema:"required"`
+	// `enabled` indicates if the provider is enabled or disabled.
+	Enabled bool `yaml:"enabled" json:"enabled,omitempty" koanf:"enabled" jsonschema:"default=false"`
+	// `scopes` is a list of scopes requested from the provider that specify the level of access an application has to
+	// a user's resources on a server, defining what actions the app can perform on behalf of the user.
+	Scopes []string `yaml:"scopes" json:"scopes" koanf:"scopes" jsonschema:"required"`
+	// `secret` is the client secret for the OAuth/OIDC client. Must be obtained from the provider.
+	//
+	// Required if the provider is `enabled`.
+	Secret string `yaml:"secret" json:"secret" koanf:"secret"`
+	// URL of the provider's token endpoint URL where an application exchanges an authorization code for an access
+	// token, which is used to authenticate API requests on behalf of the end-user.
+	//
+	// Required if `use_discovery` is false or omitted.
+	TokenEndpoint string `yaml:"token_endpoint" json:"token_endpoint,omitempty" koanf:"token_endpoint"`
+	// `use_discovery` determines if configuration information about an OpenID Connect (OIDC) provider, such as
+	// endpoint URLs and supported features,should be automatically retrieved, from a well-known
+	// URL (typically /.well-known/openid-configuration).
+	UseDiscovery bool `yaml:"use_discovery" json:"use_discovery,omitempty" koanf:"use_discovery" jsonschema:"default=true"`
+	// URL of the provider's endpoint that returns claims about an authenticated end-user.
+	//
+	// Required if `use_discovery` is false or omitted.
+	UserinfoEndpoint string `yaml:"userinfo_endpoint" json:"userinfo_endpoint,omitempty" koanf:"userinfo_endpoint"`
+}
+
+func (p *CustomThirdPartyProvider) Validate() error {
+	if p.Enabled {
+		if p.DisplayName == "" {
+			return errors.New("missing display_name")
+		}
+		if p.ClientID == "" {
+			return errors.New("missing client_id")
+		}
+		if p.Secret == "" {
+			return errors.New("missing client secret")
+		}
+		if len(p.Scopes) == 0 {
+			return errors.New("missing scopes")
+		}
+		if p.UseDiscovery == true {
+			if p.Issuer == "" {
+				return errors.New("issuer must be set when use_discovery is set to true")
+			}
+		} else {
+			authorizationEndpointSet := p.AuthorizationEndpoint != ""
+			tokenEndpointSet := p.TokenEndpoint != ""
+			userinfoEndpointSet := p.UserinfoEndpoint != ""
+
+			if !authorizationEndpointSet || !tokenEndpointSet || !userinfoEndpointSet {
+				return errors.New("authorization_endpoint, token_endpoint and userinfo_endpoint must be set when use_discovery is set to false or unset")
+			}
+		}
+	}
+
+	return nil
+}
+
+func (CustomThirdPartyProvider) JSONSchemaExtend(schema *jsonschema.Schema) {
+	schema.Title = "custom_provider"
+
+	enabledTrue := &jsonschema.Schema{Properties: orderedmap.New[string, *jsonschema.Schema]()}
+	enabledTrue.Properties.Set("enabled", &jsonschema.Schema{Const: true})
+
+	useDiscoveryFalse := &jsonschema.Schema{Properties: orderedmap.New[string, *jsonschema.Schema]()}
+	useDiscoveryFalse.Properties.Set("use_discovery", &jsonschema.Schema{Const: false})
+
+	useDiscoveryTrue := &jsonschema.Schema{Properties: orderedmap.New[string, *jsonschema.Schema]()}
+	useDiscoveryTrue.Properties.Set("use_discovery", &jsonschema.Schema{Const: true})
+
+	useDiscoveryNull := &jsonschema.Schema{Properties: orderedmap.New[string, *jsonschema.Schema]()}
+	useDiscoveryNull.Properties.Set("use_discovery", &jsonschema.Schema{Type: "null"})
+
+	useDiscoveryFalseOrNull := &jsonschema.Schema{AnyOf: []*jsonschema.Schema{useDiscoveryFalse, useDiscoveryNull}}
+
+	endpointsRequired := &jsonschema.Schema{
+		Required: []string{"authorization_endpoint", "token_endpoint", "userinfo_endpoint"},
+	}
+
+	issuerRequired := &jsonschema.Schema{
+		Required: []string{"issuer"},
+	}
+
+	schema.If = enabledTrue
+	schema.Then = &jsonschema.Schema{
+		Required: []string{"client_id", "secret"},
+		If:       useDiscoveryFalseOrNull,
+		Then:     endpointsRequired,
+		Else:     issuerRequired,
+	}
+}
+
 type ThirdPartyProviders struct {
 	// `apple` contains the provider configuration for Apple.
 	Apple ThirdPartyProvider `yaml:"apple" json:"apple,omitempty" koanf:"apple"`
@@ -181,6 +378,9 @@ type ThirdPartyProvider struct {
 	//
 	// Required if the provider is `enabled`.
 	Secret string `yaml:"secret" json:"secret,omitempty" koanf:"secret"`
+	// `name` is a unique name/slug/identifier for the provider. It is the lowercased key of the corresponding field
+	// in ThirdPartyProviders. See also: CustomThirdPartyProvider.Name.
+	Name string `jsonschema:"-"`
 }
 
 func (ThirdPartyProvider) JSONSchemaExtend(schema *jsonschema.Schema) {

backend/dto/email.go

@@ -2,6 +2,7 @@ package dto
 
 import (
 	"github.com/gofrs/uuid"
+	"github.com/teamhanko/hanko/backend/config"
 	"github.com/teamhanko/hanko/backend/persistence/models"
 )
 
@@ -23,17 +24,17 @@ type EmailUpdateRequest struct {
 }
 
 // FromEmailModel Converts the DB model to a DTO object
-func FromEmailModel(email *models.Email) *EmailResponse {
+func FromEmailModel(email *models.Email, cfg *config.Config) *EmailResponse {
 	emailResponse := &EmailResponse{
 		ID:         email.ID,
 		Address:    email.Address,
 		IsVerified: email.Verified,
 		IsPrimary:  email.IsPrimary(),
-		Identities: FromIdentitiesModel(email.Identities),
+		Identities: FromIdentitiesModel(email.Identities, cfg),
 	}
 
 	if len(email.Identities) > 0 {
-		identity := FromIdentityModel(&email.Identities[0])
+		identity := FromIdentityModel(&email.Identities[0], cfg)
 		emailResponse.Identity = identity
 	}
 

backend/dto/profile.go

@@ -37,7 +37,7 @@ func ProfileDataFromUserModel(user *models.User, cfg *config.Config) *ProfileDat
 
 	var emails []EmailResponse
 	for _, emailModel := range user.Emails {
-		email := FromEmailModel(&emailModel)
+		email := FromEmailModel(&emailModel, cfg)
 		emails = append(emails, *email)
 	}