Skip to content

Commit

Permalink
feat: mysql group replication (#1409)
Browse files Browse the repository at this point in the history
Co-authored-by: xuriwuyun <[email protected]>
  • Loading branch information
xuriwuyun and xuriwuyun committed Jan 22, 2025
1 parent 2c44983 commit 69aba21
Show file tree
Hide file tree
Showing 16 changed files with 519 additions and 41 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ KubeBlocks add-ons.
| minio | minio-2024.6.29 | High Performance, Kubernetes Native Object Storage | fengluodb |
| mogdb | mogdb-5.0.5 | A Helm chart for Kubernetes | yabinji shanshanying |
| mongodb | mongodb-4.0.28<br>mongodb-4.2.24<br>mongodb-4.4.29<br>mongodb-5.0.28<br>mongodb-6.0.16<br>mongodb-7.0.12 | MongoDB is a document database designed for ease of application development and scaling. | xuriwuyun |
| mysql | mysql-5.7.44<br>mysql-8.0.30<br>mysql-8.0.31<br>mysql-8.0.32<br>mysql-8.0.33<br>mysql-8.0.34<br>mysql-8.0.35<br>mysql-8.0.36<br>mysql-8.0.37<br>mysql-8.0.38<br>mysql-8.0.39<br>mysql-8.4.0<br>mysql-8.4.1<br>mysql-8.4.2<br>mysql-orc-5.7.44<br>mysql-orc-8.0.30<br>mysql-orc-8.0.31<br>mysql-orc-8.0.32<br>mysql-orc-8.0.33<br>mysql-orc-8.0.34<br>mysql-orc-8.0.35<br>mysql-orc-8.0.36<br>mysql-orc-8.0.37<br>mysql-orc-8.0.38<br>mysql-orc-8.0.39<br>mysql-proxysql-2.4.4 | MySQL is a widely used, open-source relational database management system (RDBMS) | xuriwuyun |
| nebula | nebula-v3.5.0 | NebulaGraph is a popular open-source graph database that can handle large volumes of data with milliseconds of latency, scale up quickly, and have the ability to perform fast graph analytics. | Shanshan Ying Xuntao Cheng |
| neon | neon-broker-1.0.0<br>neon-compute-1.0.0<br>neon-pageserver-1.0.0<br>neon-safekeeper-1.0.0 | Neon is a serverless open-source alternative to AWS Aurora Postgres. It separates storage and compute and substitutes the PostgreSQL storage layer by redistributing data across a cluster of nodes. | skyrise |
| oceanbase-ce | oceanbase-ce-4.3.0 | OceanBase has served over 400 customers across the globe and has been supporting all mission critical systems in Alipay. | Powerfool shanshanying |
| mysql | mysql-5.7.44<br>mysql-8.0.30<br>mysql-8.0.31<br>mysql-8.0.32<br>mysql-8.0.33<br>mysql-8.0.34<br>mysql-8.0.35<br>mysql-8.0.36<br>mysql-8.0.37<br>mysql-8.0.38<br>mysql-8.0.39<br>mysql-8.4.0<br>mysql-8.4.1<br>mysql-8.4.2<br>mysql-mgr-8.0.30<br>mysql-mgr-8.0.31<br>mysql-mgr-8.0.32<br>mysql-mgr-8.0.33<br>mysql-mgr-8.0.34<br>mysql-mgr-8.0.35<br>mysql-mgr-8.0.36<br>mysql-mgr-8.0.37<br>mysql-mgr-8.0.38<br>mysql-mgr-8.0.39<br>mysql-mgr-8.4.0<br>mysql-mgr-8.4.1<br>mysql-mgr-8.4.2<br>mysql-orc-5.7.44<br>mysql-orc-8.0.30<br>mysql-orc-8.0.31<br>mysql-orc-8.0.32<br>mysql-orc-8.0.33<br>mysql-orc-8.0.34<br>mysql-orc-8.0.35<br>mysql-orc-8.0.36<br>mysql-orc-8.0.37<br>mysql-orc-8.0.38<br>mysql-orc-8.0.39<br>mysql-proxysql-2.4.4 | MySQL is a widely used, open-source relational database management system (RDBMS) | xuriwuyun |
| nebula | nebula-v3.5.0 | NebulaGraph is a popular open-source graph database that can handle large volumes of data with milliseconds of latency, scale up quickly, and have the ability to perform fast graph analytics. | shanshanying Xuntao Cheng |
| neon | neon-broker-1.0.0<br>neon-compute-1.0.0<br>neon-pageserver-1.0.0<br>neon-safekeeper-1.0.0 | Neon is a serverless open-source alternative to AWS Aurora Postgres. It separates storage and compute and substitutes the PostgreSQL storage layer by redistributing data across a cluster of nodes. | skyrise-l |
| oceanbase-ce | oceanbase-ce-4.3.0 | OceanBase has served over 400 customers across the globe and has been supporting all mission critical systems in Alipay. | Powerfooi shanshanying |
| opensearch | opensearch-2.7.0<br>opensearch-dashboard-2.7.0 | Open source distributed and RESTful search engine. | iziang |
| orchestrator | orchestrator-3.2.6 | Orchestrator is a MySQL high availability and replication management tool, runs as a service and provides command line access, HTTP API and Web interface. | kubeJocker |
| orioledb | orioledb-14.7.2 | OrioleDB is a new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world's most-loved database platform. | 1aal |
Expand Down
16 changes: 7 additions & 9 deletions addons-cluster/mysql/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -1,21 +1,19 @@
{{/*
Define replica count.
standalone mode: 1
replication mode: 2
raftGroup mode: 3 or more
semisync: 2 or more
mgr: 3 or more
orchestrator mode: 2 or more
*/}}
{{- define "mysql-cluster.replicaCount" -}}
{{- if .Values.orchestrator.enable }}
replicas: {{ max .Values.replicas 2 }}
{{- else }}
{{- if eq .Values.mode "standalone" }}
replicas: 1
{{- else if eq .Values.mode "replication" }}
replicas: {{ max .Values.replicas 2 }}
{{- else }}
{{- if hasPrefix "semisync" .Values.topology }}
replicas: 2
{{- else if hasPrefix "mgr" .Values.topology }}
replicas: {{ max .Values.replicas 3 }}
{{- else }}
replicas: {{ max .Values.replicas 2 }}
{{- end }}
{{- end }}
{{- end }}
11 changes: 1 addition & 10 deletions addons-cluster/mysql/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,6 @@
"5.7.44"
]
},
"mode": {
"title": "Mode",
"description": "Cluster topology mode.",
"type": "string",
"default": "standalone",
"enum": [
"standalone",
"replication"
]
},
"replicas": {
"title": "Replicas",
"description": "The number of replicas.",
Expand Down Expand Up @@ -77,6 +67,7 @@
"enum": [
"semisync",
"semisync-proxysql",
"mgr",
"orc",
"orc-proxysql"
]
Expand Down
9 changes: 3 additions & 6 deletions addons-cluster/mysql/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,9 @@
##
version: 8.0.39

## @param mode MySQL cluster topology mode, standalone, replication
##
mode: standalone

## @param replicas specify cluster replicas
##
replicas: 1
replicas: 2

## @param cpu
##
Expand All @@ -32,7 +28,7 @@ requests:
## @param storage size, the unit is Gi
##
storage: 20
topology: semisync
topology: mgr
proxysql:
replicas: 1
cpu: 1
Expand All @@ -42,6 +38,7 @@ proxysql:
extra:
rbacEnabled: true
terminationPolicy: Delete
disableExporter: true

orchestrator:
# monitor_password: proxysql
Expand Down
8 changes: 8 additions & 0 deletions addons/mysql/config/mysql-group-replication.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[mysqld]
binlog_format=ROW
binlog_checksum=NONE
plugin_load_add='group_replication.so'
plugin_load_add='mysql_clone.so'
loose_group_replication_recovery_get_public_key=ON
loose_group_replication_start_on_boot=off
loose_group_replication_bootstrap_group=off
4 changes: 0 additions & 4 deletions addons/mysql/config/mysql5.7-config.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,6 @@ character_set_server = utf8mb4
# rpl_semi_sync_master_timeout = 1000
# rpl-semi-sync-slave-enabled = 1

[mysql]
default-character-set=utf8mb4

[client]
port={{ $mysql_port }}
socket=/var/run/mysqld/mysqld.sock
default-character-set=utf8mb4
4 changes: 0 additions & 4 deletions addons/mysql/config/mysql8.0-config.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -198,10 +198,6 @@ default_tmp_storage_engine=innodb
collation_server = utf8mb4_unicode_520_ci
character_set_server = utf8mb4

[mysql]
default-character-set=utf8mb4

[client]
port={{ $mysql_port }}
socket=/var/run/mysqld/mysqld.sock
default-character-set=utf8mb4
48 changes: 45 additions & 3 deletions addons/mysql/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,36 @@ systemAccounts:
numDigits: 5
numSymbols: 0
letterCase: MixedCases
- name: kbadmin
statement: select 1;
passwordGenerationPolicy: &defaultPasswordGenerationPolicy
length: 16
numDigits: 8
numSymbols: 0
letterCase: MixedCases
- name: kbdataprotection
statement: CREATE USER ${KB_ACCOUNT_NAME} IDENTIFIED BY '${KB_ACCOUNT_PASSWORD}';GRANT RELOAD, LOCK TABLES, PROCESS, REPLICATION CLIENT ON ${ALL_DB} TO ${KB_ACCOUNT_NAME}; GRANT LOCK TABLES,RELOAD,PROCESS,REPLICATION CLIENT, SUPER,SELECT,EVENT,TRIGGER,SHOW VIEW ON ${ALL_DB} TO ${KB_ACCOUNT_NAME};
passwordGenerationPolicy: *defaultPasswordGenerationPolicy
- name: kbprobe
statement: CREATE USER ${KB_ACCOUNT_NAME} IDENTIFIED BY '${KB_ACCOUNT_PASSWORD}'; GRANT REPLICATION CLIENT, PROCESS ON ${ALL_DB} TO ${KB_ACCOUNT_NAME}; GRANT SELECT ON performance_schema.* TO ${KB_ACCOUNT_NAME};
passwordGenerationPolicy: *defaultPasswordGenerationPolicy
- name: kbmonitoring
statement: CREATE USER ${KB_ACCOUNT_NAME} IDENTIFIED BY '${KB_ACCOUNT_PASSWORD}'; GRANT REPLICATION CLIENT, PROCESS ON ${ALL_DB} TO ${KB_ACCOUNT_NAME}; GRANT SELECT ON performance_schema.* TO ${KB_ACCOUNT_NAME};
passwordGenerationPolicy: *defaultPasswordGenerationPolicy
- name: kbreplicator
statement: select 1;
passwordGenerationPolicy: *defaultPasswordGenerationPolicy
- name: proxysql
statement: CREATE USER IF NOT EXISTS '${KB_ACCOUNT_NAME}' IDENTIFIED BY '${KB_ACCOUNT_PASSWORD}'; GRANT SELECT ON performance_schema.* TO '${KB_ACCOUNT_NAME}'; GRANT SELECT ON sys.* TO '${KB_ACCOUNT_NAME}';
vars:
- name: CLUSTER_NAME
valueFrom:
clusterVarRef:
clusterName: Required
- name: CLUSTER_UUID
valueFrom:
clusterVarRef:
clusterUID: Required
- name: CLUSTER_NAMESPACE
valueFrom:
clusterVarRef:
Expand All @@ -139,21 +162,40 @@ vars:
credentialVarRef:
name: root
password: Required
- name: MYSQL_ADMIN_USER
valueFrom:
credentialVarRef:
name: kbadmin
username: Required
- name: MYSQL_ADMIN_PASSWORD
valueFrom:
credentialVarRef:
name: kbadmin
password: Required
- name: MYSQL_REPLICATION_USER
valueFrom:
credentialVarRef:
name: kbreplicator
username: Required
- name: MYSQL_REPLICATION_PASSWORD
valueFrom:
credentialVarRef:
name: kbreplicator
password: Required
lifecycleActions:
accountProvision:
exec:
container: mysql
image: {{ .Values.image.registry | default "docker.io" }}/{{ .Values.image.repository }}:8.0.33
command:
- /bin/sh
- bash
- -c
- |
set -ex
ALL_DB='*.*'
eval statement=\"${KB_ACCOUNT_STATEMENT}\"
mysql -u${MYSQL_ROOT_USER} -p${MYSQL_ROOT_PASSWORD} -P3306 -h127.0.0.1 -e "${statement}"
targetPodSelector: Role
matchingKey: primary

roleProbe:
periodSeconds: {{ .Values.roleProbe.periodSeconds }}
timeoutSeconds: {{ .Values.roleProbe.timeoutSeconds }}
Expand Down
12 changes: 12 additions & 0 deletions addons/mysql/templates/_names.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ Define mysql orc component definition name prefix
{{ include "mysql.cmpdNamePrefix" . }}-orc
{{- end -}}

{{- define "mysql.cmpdMGRNamePrefix" -}}
{{ include "mysql.cmpdNamePrefix" . }}-mgr
{{- end -}}

{{/*
Define mysql component definition regex regular
*/}}
Expand Down Expand Up @@ -54,13 +58,21 @@ Define mysql component definition name
{{- printf "%s-8.0-%s" (include "mysql.cmpdOrcNamePrefix" .) .Chart.Version -}}
{{- end -}}

{{- define "mysql.componentDefNameMGR80" -}}
{{- printf "%s-8.0-%s" (include "mysql.cmpdMGRNamePrefix" .) .Chart.Version -}}
{{- end -}}

{{/*
Define mysql component definition name
*/}}
{{- define "mysql.componentDefName84" -}}
{{- printf "%s-8.4-%s" (include "mysql.cmpdNamePrefix" .) .Chart.Version -}}
{{- end -}}

{{- define "mysql.componentDefNameMGR84" -}}
{{- printf "%s-8.4-%s" (include "mysql.cmpdMGRNamePrefix" .) .Chart.Version -}}
{{- end -}}

{{/*
Define mysql component definition name
*/}}
Expand Down
4 changes: 4 additions & 0 deletions addons/mysql/templates/clusterdefinition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ spec:
- name: mysql
compDef: {{ include "mysql.componentDefRegex" . }}
default: true
- name: mgr
components:
- name: mysql
compDef: {{ include "mysql.cmpdMGRNamePrefix" . }}
- name: orc
components:
- name: mysql
Expand Down
120 changes: 120 additions & 0 deletions addons/mysql/templates/cmpd-mysql80-mgr.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
apiVersion: apps.kubeblocks.io/v1
kind: ComponentDefinition
metadata:
name: {{ include "mysql.componentDefNameMGR80" . }}
labels:
{{- include "mysql.labels" . | nindent 4 }}
annotations:
{{- include "mysql.annotations" . | nindent 4 }}
spec:
{{- include "mysql.spec.common" . | nindent 2 }}
serviceVersion: 8.0.33
configs:
- name: mysql-replication-config
templateRef: mysql-8.0-mgr-config-template
constraintRef: mysql-8.0-config-constraints
volumeName: mysql-config
namespace: {{ .Release.Namespace }}
reRenderResourceTypes:
- vscale
runtime:
initContainers:
- image: {{ .Values.image.registry | default "docker.io" }}/apecloud/mysql_audit_log:8.0.33
name: init-data
command:
- bash
- -c
- |
mkdir -p {{ .Values.dataMountPath }}/{log,binlog,auditlog}
cp /usr/lib/mysql/plugin/ {{ .Values.dataMountPath }}/plugin -r
imagePullPolicy: {{ default .Values.image.pullPolicy "IfNotPresent" }}
volumeMounts:
- mountPath: {{ .Values.dataMountPath }}
name: data
- command:
- cp
- -r
- /jemalloc/lib/
- /tools/lib
image: {{ .Values.image.registry | default "docker.io" }}/apecloud/jemalloc:5.3.0
imagePullPolicy: {{ default "IfNotPresent" .Values.image.pullPolicy }}
name: init-jemalloc
volumeMounts:
- mountPath: /tools
name: tools
{{- include "mysql.spec.runtime.common" . | nindent 6 }}
containers:
- name: mysql
image: {{ .Values.image.registry | default "docker.io" }}/{{ .Values.image.repository }}:8.0.33
imagePullPolicy: {{ default .Values.image.pullPolicy "IfNotPresent" }}
command:
- syncer
- --port
- "3601"
- --
- bash
- -c
- |
cp {{ .Values.dataMountPath }}/plugin/audit_log.so /usr/lib64/mysql/plugin/
chown -R mysql:root {{ .Values.dataMountPath }}
SERVICE_ID=$((${POD_NAME##*-} + 1))
docker-entrypoint.sh mysqld --server-id $SERVICE_ID \
--report-host ${POD_NAME}.${CLUSTER_COMPONENT_NAME}-headless \
--plugin-load-add=rpl_semi_sync_source=semisync_source.so \
--plugin-load-add=rpl_semi_sync_replica=semisync_replica.so \
--plugin-load-add=audit_log=audit_log.so \
--log-bin=/var/lib/mysql/binlog/${POD_NAME}-bin \
--skip-slave-start=ON
volumeMounts:
- mountPath: {{ .Values.dataMountPath }}
name: data
- mountPath: /etc/mysql/conf.d
name: mysql-config
- name: scripts
mountPath: /scripts
- mountPath: /tools
name: tools
ports:
- containerPort: 3306
name: mysql
- containerPort: 3601
name: ha
- containerPort: 33061
name: mgr
env:
- name: PATH
value: /tools/xtrabackup/bin:/tools/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
- name: LD_PRELOAD
value: /tools/lib/libjemalloc.so.2
- name: KB_ENGINE_TYPE
value: mysql
- name: KB_WORKLOAD_TYPE
value: mgr
- name: MYSQL_INITDB_SKIP_TZINFO
value: "1"
- name: MYSQL_ROOT_HOST
value: {{ .Values.auth.rootHost | default "%" | quote }}
- name: SERVICE_PORT
value: "3306"
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: POD_UID
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.uid
- name: POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: mysql-exporter
{{- include "mysql.spec.runtime.exporter" . | nindent 8 }}
Loading

0 comments on commit 69aba21

Please sign in to comment.