Can I run git2consul as a CI job?

The basics:

setup: run git2consul in a context that has been initiated with:

npm i -g git2consul pino-pretty

When you provide a single repo, and do not include any hooks - the process runs, and exits once the repo is udpated.

e.g, assuming file:

{
  "local_store": "/var/git2consul/.cache",
  "logger" : {
    "name" : "git2consul",
    "streams" : [{
      "level": "info",
      "stream": "process.stdout"
    }]
  },
 "repos" : [{
    "name" : "main",
    "url": "https://github.com/ryanbreen/git2consul_data.git",
    "branches": ["master"]
   }]
}

Then, running:

git2consul --config-file git2consul.json | pino-pretty --ignore v

runs and exits in the end of the update.

Note that the --config-file is necessary only on the first run. On consecutive runs, if the --config-file is not provided - it uses the config it finds on the consul, which must always be provided, either as CLI switches, or as environment-variables..

If you are using git2consul to pour multiple repositories to the same consul then use one of the following options:

1. provide each repo with it's own --config-key. e.g.

git2consul --config-key git2consul/repo1-config

2. delete the config it stores on consul before running git2consul, and let the config_seeder create it from scratch. You can use this method only if all your repos run in CI (i.e none of them use git2consul that runs as a service and uses hooks) e.g:

curl -X DELETE -H "X-Consul-Token: $TOKEN" http://$CONSUL_ENDPOINT:$CONSUL_PORT/v1/kv/git2consul/config

in gitlab-CI

NOTES:

1. would have been better if it did not need to checkout the repo, but use the repo found already checked out by gitlab. If anybody finds how to do that - update this page.
2. If your repo is private - you need to add creds as part of the URL or using similar technique demonstrated below

Assume this project structure:

+ consul-data (directory, with all the git2consul keys)
|  + ...
+ .gitlab-ci.yaml (file - see below)
+ git2consul.template.json (file - see below)

.gitlab-ci.yaml:

stages: [ setup ]

git2consul:
  image: node:lts
  stage: setup
  rules:
    - changes: 
       - consul-data/**/*
  variables:
    CONSUL_ENDPOINT: localhost
    CONSUL_PORT: 8500
    TOKEN: $CONSUL_TOKEN  #provided securely from hosted CI variables
  script:
    - npm i -g git2consul pino-pretty
    - sed "s/@REPO@/$CI_REPOSITORY_URL/" git2consul.template.json > git2consul.json
    - git2consul --config-file git2consul.json | pino-pretty --ignore v
  cache:
    paths: 
     - .git2consul-cache

Note: CI_REPOSITORY_URL is a gitlab-ci built-in CI variable.

git2consul.template.json:

{
  "local_store": ".git2consul-cache",
  "logger" : {
    "name" : "git2consul",
    "streams" : [{
      "level": "info",
      "stream": "process.stdout"
    }]
  },
 "repos" : [{
    "name" : "main",
    "url": "@REPO@",
    "branches": ["master"],
    "source_root": "consul-data"
   }]
}

in Jenkins

Jenkinsfile

This jenkinsfile assumes that:

1. the runner has in it's context git2consul installed
2. the project has env-variables for CONSUL_PORT, CONSUL_ENDPOINT, TOKEN
3. the project contains git2consul.json which has one repo and no hooks.

pipeline {
    agent any
    
    stages {
        
        stage('Update Consul') {
            steps {
                sh "git2consul --config-key=$CONSUL_CONFIG_KEY --config-file git2consul.json"
                //TBD: it seems to fail when trying to pipe the command. 
                //     need to find how to pipe it to `pino-pretty`
            }
        }
     
    }
}

in github actions

TBD