Allow TLS termination from ENV var (#262)

* Enable https from env vars * Pass BE https to FE * Add https urls for v3 * Add OIDC urls for v4 * Update readmes * Add https xforwar to keycloak
SciCatProject · Aug 23, 2024 · b39ce30 · b39ce30
1 parent 0c03870
commit b39ce30
Show file tree

Hide file tree

Showing 26 changed files with 143 additions and 56 deletions.
diff --git a/.env b/.env
@@ -15,3 +15,7 @@ BE_VERSION=v4
 
 ## Run SciCat services in DEV mode
 # DEV=true
+
+## Set the services HTTPS URL
+# BACKEND_HTTPS_URL=https://backend-example.com
+# FRONTEND_HTTPS_URL=https://frontend-example.com
diff --git a/README.md b/README.md
@@ -53,15 +53,16 @@ They are used when adding new services or grouping services together (and do not
 
 ### Docker compose profiles and env variables configuration options
 
-| Type    | Env key            | Value: Service/Feature                                                          | Default | Backend Compatibility | Description                                                                                                                                                            | Other impacted services |
-|---------|--------------------|---------------------------------------------------------------------------------|---------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
-| profile | `COMPOSE_PROFILES` | <li>`analysis`: jupyter<li>`search`: searchapi<li>`'*'`: jupyter,searchapi</li> | `''`    | *                     | <li>analysis: enables additional jupyter notebook with python SciCat SDK installed and example notebooks<li>search: enables a SciCat interface for standardised search |                         |
-| env     | `BE_VERSION`       | <li>`v3`: backend/v3<li>`v4`: backend/v4                                        | `v4`    | as set                | Sets the be version to use in (2) of [default setup](#default-setup) to v3                                                                                             | mongodb,frontend        |
-| env     | `JOBS_ENABLED`     | `true`: rabbitmq,archivemock,jobs feature                                       | `''`    | v3                    | Creates a rabbitmq message broker which the be posts to and the archivemock listens to. It emulates the data long-term archive/retrieve workflow                       |                         |
-| env     | `ELASTIC_ENABLED`  | `true`: elastic,elastic feature                                                 | `''`    | v4                    | Creates an elastic search service and sets the be to use it for full-text searches                                                                                     |                         |
-| env     | `LDAP_ENABLED`     | `true`: ldap auth                                                               | `''`    | *                     | Creates an LDAP service and sets the be to use it as authentication backend                                                                                            |                         |
-| env     | `OIDC_ENABLED`     | `true`: oidc auth                                                               | `''`    | *                     | Creates an OIDC identity provider and sets the be to use it as authentication backend                                                                                  |                         |
-| env     | `DEV`              | `true`: backend,frontend,searchapi,archivemock in DEV mode                      | `''`    | *                     | The SciCat services' environment is prepared to easy the [development in a standardized environment](#dev-configuration)                                               |                         |
+| Type    | Env key                     | Value: Service/Feature                                                                | Default | Backend Compatibility | Description                                                                                                                                                                   | Other impacted services |
+|---------|-----------------------------|---------------------------------------------------------------------------------------|---------|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|
+| profile | `COMPOSE_PROFILES`          | <li>`analysis`: jupyter<li>`search`: searchapi<li>`'*'`: jupyter,searchapi</li>       | `''`    | *                     | <li>analysis: enables additional jupyter notebook with python SciCat SDK installed and example notebooks<li>search: enables a SciCat interface for standardized search        |                         |
+| env     | `BE_VERSION`                | <li>`v3`: backend/v3<li>`v4`: backend/v4                                              | `v4`    | as set                | Sets the BE version to use in (2) of [default setup](#default-setup) to v3                                                                                                    | mongodb,frontend        |
+| env     | `JOBS_ENABLED`              | `true`: rabbitmq,archivemock,jobs feature                                             | `''`    | v3                    | Creates a RabbitMQ message broker which the BE posts to and the archivemock listens to. It emulates the data long-term archive/retrieve workflow                              |                         |
+| env     | `ELASTIC_ENABLED`           | `true`: elastic,elastic feature                                                       | `''`    | v4                    | Creates an elastic search service and sets the BE to use it for full-text searches                                                                                            |                         |
+| env     | `LDAP_ENABLED`              | `true`: ldap auth                                                                     | `''`    | *                     | Creates an LDAP service and sets the BE to use it as authentication backend                                                                                                   |                         |
+| env     | `OIDC_ENABLED`              | `true`: oidc auth                                                                     | `''`    | *                     | Creates an OIDC identity provider and sets the BE to use it as authentication backend                                                                                         |                         |
+| env     | `DEV`                       | `true`: backend,frontend,searchapi,archivemock in DEV mode                            | `''`    | *                     | The SciCat services' environment is prepared to ease the [development in a standardized environment](#dev-configuration)                                                      |                         |
+| env     | `<SERVICE>_HTTPS_URL`       | `<URL>`: HTTPS termination                                                            | `''`    | *                     | Requests the TLS certificate for the URL to LetsEncrypt through the [proxy](#tls-configuration)                                                                               |                         |
 
 
 After optionally setting any configuration option, one can still select the services to run as described [here](#select-the-services).
@@ -94,6 +95,12 @@ If you did not remove the volume, specified a new branch, and had any uncommited
 
 </details>
 
+#### TLS configuration
+
+You can enable TLS termination of desired services by setting the `<SERVICE>_HTTPS_URL`, by setting the full URL, including `https://`. The specified HTTPS URL will get a `letsencrypt` generated certificate through the proxy setting. For more details see the [proxy instructions](./services/proxy/README.md). After setting some URLs, the required changes in dependent services are automatically resolved, as explained for example [here](./services/frontend/README.md). Whenever possible, we use either the docker internal network or the localhost subdomains.
+
+:warning: Please make sure to set all required `<SERVICE>_HTTPS_URL` whenever enabling one, as mixing public URLs and `localhost` ones might be tricky. See, for example, what is described in the [frontend documentation](./services/frontend/README.md#enable-additional-features) and the [backend documentation](./services/backend/README.md#enable-additional-features).
+
 ### Service-specific config
 
 It can be changed whenever needing to configure a service independently from the others.
@@ -218,7 +225,7 @@ To add a new service, with advanced configuration (see the [backend](./services/
     5. if the service is another version of an existing one, e.g. v3 and v4 versions of the `backend` service, add the selective include in the parent compose.yaml, e.g. [here](./services/backend/compose.yaml)
     6. eventually, modify the [compose workflow](.github/workflows/compose_test.yaml) to add the toggle to the matrix. If the toggle depends on the changed files, remember to create the toggle configuration [here](.github/changed_files.yaml) and create the [exclude](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#excluding-matrix-configurations) rule in the workflow.
 
-4. eventually, add entrypoints for init logics, as described [here](#if-the-service-does-not-support-entrypoints-yet-one-needs-to), e.g. like [here](./services/backend/services/v4/compose.base.yaml)
+4. eventually, add entrypoints for init logics, as described [here](#if-the-service-does-not-support-entrypoints-yet-one-needs-to), e.g. like [here](./services/backend/services/v4/compose.base.yaml), including any [ENVs](#docker-compose-env-variables) specific logic. Remember to set the environment variable in the compose.yaml file. See, for example, the frontend [entrypoint](./services/frontend/entrypoints/merge_json.sh) and [compose file](./services/frontend/compose.base.yaml).
 
 </details>
 

diff --git a/services/backend/README.md b/services/backend/README.md
@@ -2,13 +2,17 @@
 
 The SciCat backend HTTP service.
 
-## Dependency on `BE_VERSION`
+## Enable additional features
 
 The `BE_VERSION` value controls which version of the backend should be started, either [v3](./services/v3) or [v4](./services/v4) (default).
 
+Setting the [BACKEND_HTTPS_URL and OIDC_ENABLED env variables](../../.env) requires changing the OIDC configuration, either in the v3 [compose.oidc.yaml](./services/v3/compose.oidc.yaml) and [providers.oidc.json](./services/v3/config/providers.oidc.json), or the v4 [env file](./services/v4/config/.oidc.env).
+
 ## Dependencies
 
-Here below we show the internal dependencies of the service, which are not already covered [here](../../README.md) (if `B` depends on `A`, then we visualize as `A --> B`). The same subdomain to service convention applies.
+Here below we show the internal dependencies of the service, which are not already covered [here](../../README.md) (if `B` depends on `A`, then we visualize it as `A --> B`). The same subdomain to service convention applies.
+
+:warning: When setting `BACKEND_HTTPS_URL` and `OIDC_ENABLED`, you might need to also set `KEYCLOAK_HTTPS_URL` to correctly resolve the login flow redirects. A more detailed explanation for [v3](https://scicatproject.github.io/documentation/Development/v3.x/OIDC.html) can be found here, and it is similar for v4.
 
 ```mermaid
 graph TD

diff --git a/services/backend/services/keycloak/.compose.https.yaml b/services/backend/services/keycloak/.compose.https.yaml
@@ -0,0 +1 @@
+../../../.empty.yaml
diff --git a/services/backend/services/keycloak/compose.base.yaml b/services/backend/services/keycloak/compose.base.yaml
@@ -0,0 +1,17 @@
+services:
+  keycloak:
+    image: quay.io/keycloak/keycloak:25.0
+    command: start-dev --import-realm --http-port=80
+    env_file: ./config/.env
+    volumes:
+      - ./healthcheck/healthcheck.sh:/healthcheck.sh
+      - ./config/facility-realm.json:/opt/keycloak/data/import/facility-realm.json:ro
+    labels:
+      - traefik.http.services.keycloak.loadbalancer.server.port=80
+    healthcheck:
+      test: bash /healthcheck.sh
+      start_period: 5s
+      interval: 30s
+      timeout: 10s
+      retries: 5
+    restart: on-failure
diff --git a/services/backend/services/keycloak/compose.https.yaml b/services/backend/services/keycloak/compose.https.yaml
@@ -0,0 +1,6 @@
+services:
+  keycloak:
+    env_file:
+      - ./config/.https.env
+    labels:
+      - traefik.http.services.keycloak.loadbalancer.passhostheader=true
diff --git a/services/backend/services/keycloak/compose.yaml b/services/backend/services/keycloak/compose.yaml
@@ -1,17 +1,4 @@
-services:
-  keycloak:
-    image: quay.io/keycloak/keycloak:25.0
-    command: start-dev --import-realm --http-port=80
-    env_file: ./config/.env
-    volumes:
-      - ./healthcheck/healthcheck.sh:/healthcheck.sh
-      - ./config/facility-realm.json:/opt/keycloak/data/import/facility-realm.json:ro
-    labels:
-      - traefik.http.services.keycloak.loadbalancer.server.port=80
-    healthcheck:
-      test: bash /healthcheck.sh
-      start_period: 5s
-      interval: 30s
-      timeout: 10s
-      retries: 5
-    restart: on-failure
+include:
+  - path:
+      - compose.base.yaml
+      - .${KEYCLOAK_HTTPS_URL:+/}compose.https.yaml
diff --git a/services/backend/services/keycloak/config/.https.env b/services/backend/services/keycloak/config/.https.env
@@ -0,0 +1,2 @@
+KC_PROXY_HEADERS=xforwarded
+PROXY_ADDRESS_FORWARDING=true
diff --git a/services/backend/services/v3/compose.oidc.yaml b/services/backend/services/v3/compose.oidc.yaml
@@ -13,3 +13,6 @@ services:
       - ./config/login-callbacks.js:/home/node/app/server/boot/login-callbacks.js
     environment:
       OIDC_ENABLED: true
+      BACKEND_URL: ${BACKEND_HTTPS_URL:-http://backend.localhost}
+      FRONTEND_URL: ${FRONTEND_HTTPS_URL:-http://localhost}
+      KEYCLOAK_URL: ${KEYCLOAK_HTTPS_URL:-http://keycloak.localhost}
diff --git a/services/backend/services/v3/config/providers.oidc.json b/services/backend/services/v3/config/providers.oidc.json
@@ -5,16 +5,16 @@
     "authScheme": "openid connect",
     "module": "/home/node/app/server/boot/oidcCompatibleStrategy",
     "authPath": "/api/v3/auth/oidc",
-    "successRedirect": "http://localhost/user",
-    "failureRedirect": "http://localhost/login",
+    "successRedirect": "${FRONTEND_URL}/user",
+    "failureRedirect": "${FRONTEND_URL}/login",
     "failureFlash": true,
-    "issuer": "http://keycloak.localhost/realms/facility",
-    "authorizationURL": "http://keycloak.localhost/realms/facility/protocol/openid-connect/auth",
-    "tokenURL": "http://keycloak.localhost/realms/facility/protocol/openid-connect/token",
-    "userInfoURL": "http://keycloak.localhost/realms/facility/protocol/openid-connect/userinfo",
+    "issuer": "${KEYCLOAK_URL}/realms/facility",
+    "authorizationURL": "${KEYCLOAK_URL}/realms/facility/protocol/openid-connect/auth",
+    "tokenURL": "${KEYCLOAK_URL}/realms/facility/protocol/openid-connect/token",
+    "userInfoURL": "${KEYCLOAK_URL}/realms/facility/protocol/openid-connect/userinfo",
     "clientID": "scicat",
     "clientSecret": "q0bOG6p616zHlz4nxvC6Ex8f6JwGY0NV",
-    "callbackURL": "http://backend.localhost/auth/oidc/callback",
+    "callbackURL": "${BACKEND_URL}/auth/oidc/callback",
     "scope": [
       "email",
       "profile",

diff --git a/services/backend/services/v3/entrypoints/merge_json.sh b/services/backend/services/v3/entrypoints/merge_json.sh
@@ -4,4 +4,5 @@ for config in $(find /config -maxdepth 1 -type f -exec basename {} \; | cut -d '
 do
     # shellcheck disable=SC2016
     npx --yes [email protected] -s 'reduce .[] as $item ({}; . * $item)' /config/"${config}"*.json > /home/node/app/server/"${config}".json
+    npx --yes envsub /home/node/app/server/"${config}".json
 done
diff --git a/services/backend/services/v4/.compose.https.yaml b/services/backend/services/v4/.compose.https.yaml
@@ -0,0 +1 @@
+../../../.empty.yaml
diff --git a/services/backend/services/v4/compose.dev.yaml b/services/backend/services/v4/compose.dev.yaml
@@ -9,8 +9,8 @@ services:
       GITHUB_REPO: https://github.com/SciCatProject/scicat-backend-next.git
     volumes:
       - v4_dev:/home/node/app
-      - ${PWD}/entrypoints/setup_git.sh:/docker-entrypoints/1.sh
-      - ./entrypoints/tests.sh:/docker-entrypoints/2.sh
+      - ${PWD}/entrypoints/setup_git.sh:/docker-entrypoints/2.sh
+      - ./entrypoints/tests.sh:/docker-entrypoints/3.sh
 
 volumes:
   v4_dev:

diff --git a/services/backend/services/v4/compose.https.yaml b/services/backend/services/v4/compose.https.yaml
@@ -0,0 +1,5 @@
+services:
+  backend:
+    volumes:
+      # wait for kecyloak to get certificate
+      - ./entrypoints/sleep.sh:/docker-entrypoints/1.sh
diff --git a/services/backend/services/v4/compose.yaml b/services/backend/services/v4/compose.yaml
@@ -3,3 +3,4 @@ include:
       - compose.base.yaml
       - .${ELASTIC_ENABLED:+/}compose.elastic.yaml
       - .${DEV:+/}compose.dev.yaml
+      - .${KEYCLOAK_HTTPS_URL:+/}compose.https.yaml
diff --git a/services/backend/services/v4/config/.oidc.env b/services/backend/services/v4/config/.oidc.env
@@ -1,8 +1,8 @@
-OIDC_ISSUER=http://keycloak.localhost/realms/facility
+OIDC_ISSUER=${KEYCLOAK_HTTPS_URL:-http://keycloak.localhost}/realms/facility
 OIDC_CLIENT_ID=scicat
 OIDC_CLIENT_SECRET=q0bOG6p616zHlz4nxvC6Ex8f6JwGY0NV
-OIDC_CALLBACK_URL=http://backend.localhost/api/v3/auth/oidc/callback
+OIDC_CALLBACK_URL=${BACKEND_HTTPS_URL:-http://backend.localhost}/api/v3/auth/oidc/callback
 OIDC_SCOPE=openid profile email
-OIDC_SUCCESS_URL=http://localhost/login
+OIDC_SUCCESS_URL=${FRONTEND_HTTPS_URL:-http://localhost}/login
 ACCESS_GROUPS_OIDCPAYLOAD_ENABLED=true
 OIDC_ACCESS_GROUPS_PROPERTY=accessGroups
diff --git a/services/backend/services/v4/entrypoints/sleep.sh b/services/backend/services/v4/entrypoints/sleep.sh
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+sleep 60
diff --git a/services/frontend/README.md b/services/frontend/README.md
@@ -21,3 +21,7 @@ For an explanation of how setting `BE_VERSION` changes the environment creation
 Since there was a small breaking change from `v3` to `v4`, when connecting to the `backend`, the `BE_VERSION` value controls if [config.v3.json file](./config/config.v3.json), which is applied when `BE_VERSION=v3`, should be included in the configs merge process.
 
 With `DEV=true`, please use `npm start -- --host 0.0.0.0`. This is to allow traffic from any IP to the `frontend` component and it is necessary since the component runs in the docker network.
+
+Setting the [BACKEND_HTTPS_URL env variable](../../.env) requires changing the `backend` URL used by the `frontend`. This is managed [here](./entrypoints/merge_json.sh).
+
+:warning: When setting `FRONTENT_HTTPS_URL` it is likely you also want to set the `BACKEND_HTTPS_URL`, to allow the communication between the two wherever the browser is accessed.
diff --git a/services/frontend/compose.base.yaml b/services/frontend/compose.base.yaml
@@ -17,3 +17,5 @@ services:
     labels:
       - traefik.http.routers.frontend.rule=Host(`localhost`)
     restart: on-failure
+    environment:
+      BACKEND_URL: ${BACKEND_HTTPS_URL:-http://backend.localhost}
diff --git a/services/frontend/config/config.json b/services/frontend/config/config.json
@@ -15,7 +15,7 @@
    "jsonMetadataEnabled":true,
    "jupyterHubUrl":"",
    "landingPage":"",
-   "lbBaseURL":"http://backend.localhost",
+   "lbBaseURL":"${BACKEND_URL}",
    "localColumns":[
       {
          "name":"datasetName",

diff --git a/services/frontend/entrypoints/merge_json.sh b/services/frontend/entrypoints/merge_json.sh
@@ -1,5 +1,6 @@
 #!/bin/sh
 
-apk update && apk add jq
+apk update && apk add jq gettext
 
-jq -s 'reduce .[] as $item ({}; . * $item)' /config/*.json > "${CONFIG_DIR:-/usr/share/nginx/html/assets}"/config.json
+jq -s 'reduce .[] as $item ({}; . * $item)' /config/*.json | envsubst \
+    > "${CONFIG_DIR:-/usr/share/nginx/html/assets}"/config.json
diff --git a/services/proxy/README.md b/services/proxy/README.md
@@ -16,15 +16,4 @@ You need to set the letsencrypt options [here](./config/.tls.env).
 
 ## Enable TLS
 
-The proxy sets a default certificate resolver, using letsencrypt. To use it, you should:
-
-1. change the [resolver settings](./config/.tls.env). If the docker volume `letsencrypt_proxy_data` already exists, you might need to remove it to apply changes from the [.tls.env file](./config/.tls.env)
-2. add dedicated labels to each service to expose, making sure that the URLs are reachable by letsencrypt. You should set: the service public URL and the certificate resolver annotation and set the entrypoint to `websecure` to use port 443 only. For example, for the [frontend service](../frontend/compose.base.yaml):
-```diff
-    labels:
--     - traefik.http.routers.frontend.rule=Host(`localhost`)
-+     - traefik.http.routers.frontend.rule=Host(`<YOUR_PUBLIC_HOST>`)
-+     - traefik.http.routers.frontend.entrypoints=websecure
-```
-3. Change any other service that referenced the changed host
-4. rerun `docker compose up -d`
+To enable TLS on specific services, you can set the `<SERVICE>_HTTPS_URL` env var to the desired URL, including the `https://` prefix, making sure that the URLs are reachable by `letsencrypt`. See [here](../../.env) for an example. This will request the certificate from `letsencrypt`.
diff --git a/services/proxy/compose.yaml b/services/proxy/compose.yaml
@@ -7,12 +7,24 @@ services:
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
       - letsencrypt_proxy_data:/letsencrypt
+      - ./healthcheck/https_services.sh:/https_services.sh
+      - ./config/tls.yaml:/config/traefik.pre.yaml
     env_file:
       - ./config/.env
       - ./config/.tls.env
+      - ${PWD}/.env
     labels:
       - traefik.http.services.proxy.loadbalancer.server.port=8080
     restart: on-failure
+    command: sh -c 'touch /config/traefik.yaml; traefik'
+    environment:
+      - COMPOSE_PROJECT_NAME
+    healthcheck:
+      test: /https_services.sh
+      interval: 10s
+      timeout: 30s
+      retries: 5
+      start_period: 5s
 
 volumes:
   letsencrypt_proxy_data:

diff --git a/services/proxy/config/.env b/services/proxy/config/.env
@@ -4,6 +4,8 @@ TRAEFIK_PROVIDERS_DOCKER=true
 TRAEFIK_PROVIDERS_DOCKER_CONSTRAINTS=Label(`com.docker.compose.project`, `${COMPOSE_PROJECT_NAME}`)
 TRAEFIK_PROVIDERS_DOCKER_DEFAULTRULE=Host(`{{ normalize .Name | replace "-${COMPOSE_PROJECT_NAME}" ".localhost" }}`)
 
+TRAEFIK_PROVIDERS_FILE_FILENAME=/config/traefik.yaml
+
 TRAEFIK_ENTRYPOINTS_WEB_ADDRESS=:80
 TRAEFIK_ENTRYPOINTS_WEB_ASDEFAULT=true