Self-Hosted-Renovate: Env. Secrets usage in config.js

[dschunack]

Hi,

we use the helm chart to run a Self-Hosted renovate. At the moment we are working to integrate the access on our internal Artifactory to check for new Images, but we struggle a little bit with the configuration of the config.js file.

We configured the secrets block in the helm chart like this:

secrets:
  ARTIFACTORY_USER: XXXXXX
  ARTIFACTORY_PW: XXXXXX

The renovate-secret is still there and the secret variables are available in the POD and env. variables.

The renovate.json in the REPO looks like this:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "hostRules": [
    {
      "matchHost": "our-container-repo",
      "hostType": "docker",
      "username": "{{ secrets.ARTIFACTORY_USER }}",
      "password": "{{ secrets.ARTIFACTORY_PW }}"
    }
  ]
}

If we now start a Renovate run the secret variable is not used and it end with the following errror:

INFO: Repository has invalid config (repository=test-container)
       "error": {
         "validationSource": "config",
         "validationError": "Unknown secret name",
         "validationMessage": "The following secret name was not found in config: ARTIFACTORY_USER",
         "message": "config-validation",

We want to configure the module.export part in the config.js, but it's not working and the env variables are not used.
It end with json syntax errors or the "process.env.****" string is used as User/PW and the the env variable.

{
  "dryRun": false,
  "printConfig": false,
  "logLevel": "info",
  "requiredStatusChecks": null,
  "gitAuthor": "xxxxxxxxxxxxxxxxxxxxx",
  "platform": "github",
  "token": "xxxxxxxxxxxxxxxxxxx",
  "autodiscover": true,
  "endpoint": "https://xxxxxxxxxxxxxxxx/api/v3/",
  "autodiscoverFilter": "@(xxxxxxxxx|xxxxxxxxxxx)/*"
  "secrets": {
    "ARTIFACTORY_USER": "process.env.ARTIFACTORY_USER",
    "ARTIFACTORY_PW": "process.env.ARTIFACTORY_USER",
  }
}

If we use module.export:

DEBUG: Using RE2 as regex engine
DEBUG: Parsing configs
DEBUG: Checking for config file in /usr/src/app/config.json
FATAL: Could not parse config file 
        SyntaxError: JSON5: invalid character 'm' at 1:1
           at syntaxError (/usr/src/app/node_modules/json5/lib/parse.js:1083:17)
           at invalidChar (/usr/src/app/node_modules/json5/lib/parse.js:1028:12)
           at Object.value (/usr/src/app/node_modules/json5/lib/parse.js:287:15)
           at lex (/usr/src/app/node_modules/json5/lib/parse.js:78:42)
           at Object.parse (/usr/src/app/node_modules/json5/lib/parse.js:25:17)
           at getParsedContent (/usr/src/app/node_modules/renovate/lib/workers/global/config/parse/file.ts:20:20)
           at Object.getConfig (/usr/src/app/node_modules/renovate/lib/workers/global/config/parse/file.ts:52:14)
           at parseConfigs (/usr/src/app/node_modules/renovate/lib/workers/global/config/parse/index.ts:21:22)
           at Object.start (/usr/src/app/node_modules/renovate/lib/workers/global/index.ts:108:14)
           at /usr/src/app/node_modules/renovate/lib/renovate.ts:16:22
}

Helm chart: renovate-32.9.3
Helm config:

cronjob:
  schedule: "@hourly"
  annotations: {}
  labels: {}
  concurrencyPolicy: "Forbid"
  failedJobsHistoryLimit: ""
  successfulJobsHistoryLimit: ""
  jobRestartPolicy: "Never"

pod:
  annotations: {}
  labels: {}

resources:
  requests:
    memory: 2Gi
  limits:
    memory: 2.5Gi

env:
  NODE_OPTIONS: "--max-old-space-size=2048"

renovate:
  # See https://docs.renovatebot.com/self-hosted-configuration
  config: |
    {
      "dryRun": false,
      "printConfig": false,
      "logLevel": "info",
      "requiredStatusChecks": null,
      "gitAuthor": "xxxxxxxxxxxxxxx",
      "platform": "github",
      "token": "xxxxxxxxxxxxxxx",
      "secrets": {
        "ARTIFACTORY_USER": "process.env.ARTIFACTORY_USER",
        "ARTIFACTORY_PW": "process.env.ARTIFACTORY_PW",
      },
      "autodiscover": true,
      "endpoint": "https://xxxxxxxxxxxxxxx/api/v3/",
      "autodiscoverFilter": "@(xxxxxxxxxxxxxxx)/*container"
    }

secrets:
  ARTIFACTORY_USER: xxxxxxxxxxxxxxx
  ARTIFACTORY_PW: xxxxxxxxxxxxxxx

We tried some different spellings and variants to configure it, but none was working.
The only working way was to add the User/PW directly in the config.js instead to use "process.env.ARTIFACTORY_*", but this is not really secure and we don't wont to use it.

Maybe, someone here know the answer :-)

[marsdalesa]

@dschunack I encountered the same difficulty as you and while I didn't make it work within using an inline renovate.config I did manage to make this work by exploiting extraConfigmaps, extraVolumes, extraVolumeMounts and setting existingConfigFile. This way, I was able to make a config.js that can use the renovate docs examples. The k8s secret renovate-env contains the secret values e.g.:

apiVersion: v1
kind: Secret
metadata:
  name: renovate-env
type: Opaque
stringData:
  GITHUB_COM_TOKEN: "xxxxxx"
  RENOVATE_AUTODISCOVER: 'false'
  RENOVATE_GIT_AUTHOR: 'Renovate Bot <xxxx@xxxxx>'
  RENOVATE_TOKEN: "xxxxxx"
  ARTIFACTORY_USER: "xxxxxx"
  ARTIFACTORY_TOKEN: "xxxxx"

The helm values I set for testing were:

existingSecret: renovate-env
extraConfigmaps:
  - name: config
    data:
      config.js: |
        module.exports = {
          platform: "github",
          repositories: ["xxxx/xxxxx"],
          hostRules: [
            {
              matchHost: "https://xxxx.jfrog.io/xxxx/xxxxx-charts/",
              username: process.env.ARTIFACTORY_USER,
              password: process.env.ARTIFACTORY_TOKEN
            }
          ],
          secrets: {
            ARTIFACTORY_TOKEN: process.env.ARTIFACTORY_TOKEN,
            ARTIFACTORY_USER: process.env.ARTIFACTORY_USER
          }
        }
extraVolumes:
  - name: renovate-extra-config
    configMap:
      name: renovate-extra-config
extraVolumeMounts:
  - name: renovate-extra-config
    mountPath: /usr/src/app/config.js
    subPath: config.js
renovate:
  existingConfigFile: "/usr/src/app/config.js"

With the above you can set repo configs to use the secrets in secrets: { or rely on the bot config above to handle defaults.

[viceice]

Closing this issue, as it works as intended. By default we use a config.json. So you need a manual config like above.

Maybe @marsdalesa Can open a PR to add this sample to the readme ?

[fllaca]

Adding customizations to config.js as @marsdalesa and @dschunack need seems quite a common use case when self-hosting Renovate. @viceice do you think that @marsdalesa manual volumes setup could be adopted by the helm chart and managed with a new variable (something like renovate.custom_config.js) ?.

Example:

renovate:
  # See https://docs.renovatebot.com/self-hosted-configuration
  config: |
    {
      . . . 
    }
  # when custom_config.js is set, it creates a configmap with its content and mount it on /usr/src/app/config.js
  custom_config.js: |
    module.exports = {
      platform: "github",
      repositories: ["xxxx/xxxxx"],
      hostRules: [
        {
          matchHost: "https://xxxx.jfrog.io/xxxx/xxxxx-charts/",
          username: process.env.ARTIFACTORY_USER,
          password: process.env.ARTIFACTORY_TOKEN
        }
      ],
      secrets: {
        ARTIFACTORY_TOKEN: process.env.ARTIFACTORY_TOKEN,
        ARTIFACTORY_USER: process.env.ARTIFACTORY_USER
      }
    }

[viceice]

sure, but i prefer renovate.configjs as key.