Skip to content

Commit

Permalink
Merge pull request #107 from bcgov/helmMetabase
Browse files Browse the repository at this point in the history
Helm metabase
  • Loading branch information
thegentlemanphysicist authored Apr 21, 2022
2 parents 85d1d28 + cd3e0c4 commit 018c162
Show file tree
Hide file tree
Showing 18 changed files with 609 additions and 2 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on: push

jobs:
pre-commit:
runs-on: ubuntu-18.04
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- name: Install asdf
Expand All @@ -27,7 +27,7 @@ jobs:
asdf reshim
pre-commit run --color=always --show-diff-on-failure --all-files
commitlint:
runs-on: ubuntu-18.04
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
with:
Expand Down
20 changes: 20 additions & 0 deletions helm/backup-storage/values-eb75ad-tools-metabase-backup.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# The command to restore the latest backup is:
# ./backup.sh -r metabase-postgresql:5432/metabase
# The command to update the backup in sandbox dev is:
# make install NAME=metabase-backup NAMESPACE=eb75ad-tools
nameOverride: "metabase-backup-storage"
fullnameOverride: "metabase-backup-storage"

backupConfig: |
postgres=metabase-postgresql:5432/metabase
0 1 * * * default ./backup.sh -s
0 4 * * * default ./backup.sh -s -v all
db:
secretName: metabase-postgresql-credentials
usernameKey: username
passwordKey: password

persistence:
backup:
size: 1Gi
23 changes: 23 additions & 0 deletions helm/metabase/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
24 changes: 24 additions & 0 deletions helm/metabase/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: v2
name: metabase
description: A Helm chart for metabase

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
19 changes: 19 additions & 0 deletions helm/metabase/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
SHELL := /usr/bin/env bash

NAMESPACE=""
# This installs metabase in the gold production tools namespace
.PHONY: install
install:
helm install metabase . -n ${NAMESPACE} \
-f values.yaml \
-f "values-${NAMESPACE}.yaml" \
--set postgresql.password="$(shell bash -c 'echo $$RANDOM | md5sum | head -c 32')"

# Upgrade will change an existing metabase deployment. The secret gets regenerated each time this is run
.PHONY: upgrade
upgrade:
helm upgrade --install metabase . \
-n ${NAMESPACE} \
-f values.yaml \
-f "values-${NAMESPACE}.yaml" \
--set postgresql.password="$(shell bash -c 'echo $$RANDOM | md5sum | head -c 32')"
105 changes: 105 additions & 0 deletions helm/metabase/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Metabase config docs

## Installing metabase

### Confirm network policies

In order for metabase to deploy successfully, pods within the namespace must have access to each other. Ensure the the following two network policies are in place:

```
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-same-namespace
namespace: <NAMESPACE>
spec:
podSelector: {}
ingress:
- from:
- podSelector: {}
policyTypes:
- Ingress
```
and

```
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-from-openshift-ingress
namespace: <NAMESPACE>
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
network.openshift.io/policy-group: ingress
policyTypes:
- Ingress
```

These network policies are fairly generic and may or may not already be configured in the namespace where Metabase is being deployed.

### Install the helm charts

The two helm commands are run through the make file in this folder:

`make install NAMESPACE=eb75ad-tools`

and

`make upgrade NAMESPACE=eb75ad-tools`

This will install and upgrade metabase in the Gold Production Tools namespace. If we need to install it somewhere else the script will need to be generalized. Note this helm chart has not been used to deploy the Silver cluster metabase instance.

## Adding a databse to metabase:

The Gold Metabase deployment can query any db in the gold cluster if the credentials are set up properly. In future it may extend to silver as well. The gold configuration requires two steps:

### Set Network policies

Metabase needs to access other namespaces. To allow access we must create a network policy in the namespace holding the target database. For the sso-patroni databases, this network policy is:

```
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: <NAMESPACE DESCRIPTION>-metabase-access
namespace: <TARGET NAMESPACE>
spec:
podSelector:
matchLabels:
app: <DATABASE CLUSTER TAG>
ingress:
- from:
- namespaceSelector:
matchLabels:
environment: <METABASE NAMESPACE ENVIRONMENT>
name: <METABASE NAMESPACE LICENCE PLATE>
- podSelector:
matchLabels:
app: metabase
policyTypes:
- Ingress
```

The config for Gold Metabase are:

```
DATABASE CLUSTER TAG: sso-patroni
METABASE NAMESPACE ENVIRONMENT: tools
METABASE NAMESPACE LICENCE PLATE: eb75ad
```


### Connect to the database

Metabase has a built in workflow for adding new databases. These will require the credentials for the target database.
The one complicated part of adding a db to the metabase account is the host. This follows the patern:

```
<openshift_service_name>.<target_namespace_name>.svc.cluster.local
```

It is important to connect to the read only service if one is available. If the connection attempt times out, it is possible the network policy you created is not set up properly.
3 changes: 3 additions & 0 deletions helm/metabase/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
1. Get the application URL by running the command:

oc -n {{.Values.namespace}} get routes
65 changes: 65 additions & 0 deletions helm/metabase/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "metabase.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "metabase.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "metabase.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "metabase.labels" -}}
app: metabase
service: {{ .Values.metabaseServiceName }}
template: metabase-postgresql-template
helm.sh/chart: {{ include "metabase.chart" . }}
{{ include "metabase.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "metabase.selectorLabels" -}}
app.kubernetes.io/name: {{ include "metabase.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "metabase.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "metabase.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
Loading

0 comments on commit 018c162

Please sign in to comment.