Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs and example for multi-level cluster setup #972

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions config/examples/multi_level_cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Minimal example for multi-level cluster setup with operator
# (https://docs.victoriametrics.com/cluster-victoriametrics/#multi-level-cluster-setup)
# There are two clusters: blue and green. Each cluster has its own storage, select and insert nodes.
# And there is a shared VMCluster that uses VMSelect and VMInsert nodes from blue and green clusters through clusterNative.
# The shared cluster also has a VMAgent that sends data to the shared insert nodes and VMAuth that selects data from the shared select nodes.

---
apiVersion: v1
kind: Namespace
metadata:
name: monitoring

---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: blue
namespace: monitoring
spec:
retentionPeriod: "1"
vmstorage:
replicaCount: 3
vminsert:
replicaCount: 1
extraArgs:
clusternativeListenAddr: ":8400"
serviceSpec:
useAsDefault: true
spec:
ports:
- name: clusternative
port: 8400
protocol: TCP
targetPort: 8400
vmselect:
replicaCount: 1
extraArgs:
clusternativeListenAddr: ":8401"
serviceSpec:
useAsDefault: true
spec:
ports:
- name: clusternative
port: 8401
protocol: TCP
targetPort: 8401

---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: green
namespace: monitoring
spec:
retentionPeriod: "1"
vmstorage:
replicaCount: 3
vminsert:
replicaCount: 1
extraArgs:
clusternativeListenAddr: ":8400"
serviceSpec:
useAsDefault: true
spec:
ports:
- name: clusternative
port: 8400
protocol: TCP
targetPort: 8400
vmselect:
replicaCount: 1
extraArgs:
clusternativeListenAddr: ":8401"
serviceSpec:
useAsDefault: true
spec:
ports:
- name: clusternative
port: 8401
protocol: TCP
targetPort: 8401

---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
name: shared
namespace: monitoring
spec:
retentionPeriod: "1"
vmselect:
replicaCount: 1
extraArgs:
storageNode: "blue/vmselect-blue-0.vmselect-blue,green/vmselect-green-0.vmselect-green"
vminsert:
replicaCount: 1
extraArgs:
storageNode: "blue/vminsert-blue-0.vminsert-blue,green/vminsert-green-0.vminsert-green"

---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
name: shared
namespace: monitoring
spec:
selectAllByDefault: true
remoteWrite:
- url: "http://vminsert-shared:8480/insert/0/prometheus/api/v1/write"
- url: "http://vminsert-shared:8480/insert/0/prometheus/api/v1/write"

---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAuth
metadata:
name: shared
namespace: monitoring
spec:
userSelector: {}
userNamespaceSelector:
matchLabels:
kubernetes.io/metadata.name: monitoring
ingress:
class_name: nginx
host: vm.k8s.orb.local # specify your ingress host here
unauthorizedAccessConfig:
- src_paths: [".*"]
url_prefix: ["http://vmselect-shared:8481/select/0"]
6 changes: 6 additions & 0 deletions docs/resources/vmcluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,12 @@ Possible configuration options for backup crd can be found at [link](../api.md#v

Also see VMCLuster example spec [here](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmcluster_with_backuper.yaml).

## Multi-level cluster setup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a picture here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll add picture


You can construct a [multi-level cluster setup](https://docs.victoriametrics.com/cluster-victoriametrics/#multi-level-cluster-setup) with several `VMCluster` resources.

See minimal configuration [example here](https://github.com/VictoriaMetrics/operator/blob/master/config/examples/vmagent-stream-aggr.yaml) for that.

## Examples

### Minimal example without persistence
Expand Down
Loading