Kubernetes 部署 MySQL 主从集群（ZooKeeper 选举）
本文档介绍在 Kubernetes 环境中使用 ZooKeeper 进行主从选举的 MySQL 高可用部署方案。
架构介绍
架构概述
本方案采用 MySQL 主从复制架构，配合 ZooKeeper 实现自动主节点选举。当主节点故障时，ZooKeeper 可以协调从节点进行选举，确保服务的高可用性。
┌─────────────────────────────────────────────────────────────┐
│ Kubernetes Cluster │
│ │
│ ┌────────────────────┐ ┌────────────────────┐ │
│ │ MySQL Master │◄──►│ MySQL Replica │ │
│ │ (Pod/StatefulSet) │ │ (Pod/StatefulSet) │ │
│ └────────┬───────┘ └────────┬───────┘ │
│ │ │ │
│ ┌────────▼────────┐ ┌───▼────────────┐ │
│ │ PVC (数据) │ │ PVC (数据) │ │
│ └─────────────┘ └──────────────┘ │
└────────────────────────────────────┘
│
┌──────────┴──────────┐
│ │
┌─────────▼─────────┐ ┌────▼─────────┐
│ ZooKeeper-1 │◄─►│ ZooKeeper-3 │
│ Pod │ │ Pod │
└────────────────┘ └───────────┘
│ ◄─────────► │
Master Election Leader技术选型
- MySQL 模式：主从复制 + 半同步复制
- 选举协调：ZooKeeper（EPHEMERAL_SEQUENTIAL 节点）
- 健康检查：MySQL 健康检查 + ZooKeeper 心跳
- 故障转移：ZooKeeper 触发选举，应用层感知新主节点
为什么使用 ZooKeeper
ZooKeeper 在分布式系统中提供可靠的协调服务：
- 原子广播：ZAB 协议保证一致性
- 临时节点：自动检测节点失效
- 顺序节点：支持领导者选举
- 观察者模式：实时感知配置变化
部署资源清单
1. 命名空间
yaml
apiVersion: v1
kind: Namespace
metadata:
name: mysql-mgr
labels:
name: mysql-mgr
environment: production2. ZooKeeper 集群配置
2.1 ConfigMap
yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: zk-config
namespace: mysql-mgr
data:
zoo.cfg: |
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data
clientPort=2181
admin.enableServer=false
4lw.commands.whitelist=*
skipACL=yes
metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
metricsProvider.httpPort=7000
metricsProvider.exportJvmInfo=true
---
apiVersion: v1
kind: ConfigMap
metadata:
name: zk-log4j
namespace: mysql-mgr
data:
log4j.properties: |
zookeeper.root.logger=CONSOLE
zookeeper.console.threshold=INFO2.2 ZooKeeper Service
yaml
apiVersion: v1
kind: Service
metadata:
name: zookeeper-headless
namespace: mysql-mgr
spec:
clusterIP: None
ports:
- name: client
port: 2181
- name: quorum
port: 2888
- name: leader-election
port: 3888
selector:
app: zookeeper
---
apiVersion: v1
kind: Service
metadata:
name: zookeeper
namespace: mysql-mgr
spec:
type: ClusterIP
ports:
- name: client
port: 2181
targetPort: 2181
selector:
app: zookeeper2.3 ZooKeeper StatefulSet
yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zookeeper
namespace: mysql-mgr
spec:
serviceName: zookeeper-headless
replicas: 3
selector:
matchLabels:
app: zookeeper
template:
metadata:
labels:
app: zookeeper
spec:
terminationGracePeriodSeconds: 30
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: zookeeper
topologyKey: kubernetes.io/hostname
containers:
- name: zookeeper
image: zookeeper:3.9.2
imagePullPolicy: IfNotPresent
ports:
- name: client
containerPort: 2181
- name: quorum
containerPort: 2888
- name: leader-election
containerPort: 3888
env:
- name: ZOO_MY_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: ZOO_SERVERS
value: "0=zookeeper-0.zookeeper-headless.mysql-mgr.svc.cluster.local:2888:3888,1=zookeeper-1.zookeeper-headless.mysql-mgr.svc.cluster.local:2888:3888,2=zookeeper-2.zookeeper-headless.mysql-mgr.svc.cluster.local:2888:3888"
- name: JVMFLAGS
value: "-Xmx512m -Xms512m"
resources:
requests:
cpu: 100m
memory: 512Mi
limits:
cpu: 500m
memory: 1Gi
volumeMounts:
- name: data
mountPath: /data
- name: config
mountPath: /conf
livenessProbe:
exec:
command:
- zookeeper-shell
- localhost:2181
- ruok
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
exec:
command:
- zookeeper-shell
- localhost:2181
- ruok
initialDelaySeconds: 10
periodSeconds: 5
volumes:
- name: data
persistentVolumeClaim:
claimName: zookeeper-data
- name: config
configMap:
name: zk-config
volumeClaimTemplates:
- metadata:
name: zookeeper-data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: fast
resources:
requests:
storage: 5Gi3. MySQL 配置
3.1 ConfigMap
yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: mysql-config
namespace: mysql-mgr
data:
my.cnf: |
[mysqld]
# 基础配置
default-storage-engine=InnoDB
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
max_connections=2000
# 网络配置
bind-address=0.0.0.0
port=3306
# GTID 配置
gtid_mode=ON
enforce_gtid_consistency=ON
# 二进制日志
log-bin=mysql-bin
binlog_format=ROW
log_replica_updates=ON
expire_logs_days=7
max_binlog_size=256M
# 中继日志
relay_log=relay-bin
relay_log_recovery=ON
# 复制配置
rpl_semi_sync_source_enabled=ON
rpl_semi_sync_replica_enabled=ON
replica_skip_errors=1062
# 只读配置
read_only=1
super_read_only=1
# 优化配置
innodb_buffer_pool_size=1G
innodb_log_file_size=256M
innodb_flush_log_at_trx_commit=1
table_open_cache=2000
# 复制同步超时
rpl_semi_sync_source_timeout=1000
rpl_semi_sync_replica_timeout=1000
# 日志
log_error=/var/log/mysql/error.log
slow_query_log=1
slow_query_log_file=/var/log/mysql/slow.log
long_query_time=2
[client]
default-character-set=utf8mb4
[mysql]
default-character-set=utf8mb43.2 MySQL 主节点 Service
yaml
apiVersion: v1
kind: Service
metadata:
name: mysql-master-headless
namespace: mysql-mgr
spec:
clusterIP: None
ports:
- name: mysql
port: 3306
selector:
app: mysql
role: master
---
apiVersion: v1
kind: Service
metadata:
name: mysql-master
namespace: mysql-mgr
spec:
type: ClusterIP
ports:
- name: mysql
port: 3306
targetPort: 3306
selector:
app: mysql
role: master3.3 MySQL 从节点 Service
yaml
apiVersion: v1
kind: Service
metadata:
name: mysql-replica-headless
namespace: mysql-mgr
spec:
clusterIP: None
ports:
- name: mysql
port: 3306
selector:
app: mysql
role: replica
---
apiVersion: v1
kind: Service
metadata:
name: mysql-replica
namespace: mysql-mgr
spec:
type: ClusterIP
ports:
- name: mysql
port: 3306
targetPort: 3306
selector:
app: mysql
role: replica4. MySQL StatefulSet
4.1 主节点 StatefulSet
yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql-master
namespace: mysql-mgr
spec:
serviceName: mysql-master-headless
replicas: 1
selector:
matchLabels:
app: mysql
role: master
template:
metadata:
labels:
app: mysql
role: master
spec:
terminationGracePeriodSeconds: 30
containers:
- name: mysql
image: mysql:8.0.36
imagePullPolicy: IfNotPresent
ports:
- name: mysql
containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secrets
key: root-password
- name: MYSQL_REPLICATION_USER
value: "repl"
- name: MYSQL_REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secrets
key: repl-password
- name: ZOOKEEPER_SERVERS
value: "zookeeper.mysql-mgr.svc.cluster.local:2181"
- name: SERVER_ID
value: "1"
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
memory: 2Gi
volumeMounts:
- name: mysql-data
mountPath: /var/lib/mysql
- name: mysql-config
mountPath: /etc/mysql/conf.d
- name: mysql-logs
mountPath: /var/log/mysql
livenessProbe:
exec:
command:
- mysqladmin
- ping
- -h
- localhost
- -uroot
- -pRoot@2024
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
exec:
command:
- mysql
- -h
- localhost
- -uroot
- -pRoot@2024
- -e
- SELECT 1
initialDelaySeconds: 30
periodSeconds: 5
volumes:
- name: mysql-data
persistentVolumeClaim:
claimName: mysql-master-data
- name: mysql-config
configMap:
name: mysql-config
- name: mysql-logs
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: mysql-master-data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: fast
resources:
requests:
storage: 50Gi4.2 从节点 StatefulSet
yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql-replica
namespace: mysql-mgr
spec:
serviceName: mysql-replica-headless
replicas: 2
selector:
matchLabels:
app: mysql
role: replica
template:
metadata:
labels:
app: mysql
role: replica
spec:
terminationGracePeriodSeconds: 30
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: mysql
role: replica
topologyKey: kubernetes.io/hostname
containers:
- name: mysql
image: mysql:8.0.36
imagePullPolicy: IfNotPresent
ports:
- name: mysql
containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secrets
key: root-password
- name: MYSQL_REPLICATION_USER
value: "repl"
- name: MYSQL_REPLICATION_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secrets
key: repl-password
- name: ZOOKEEPER_SERVERS
value: "zookeeper.mysql-mgr.svc.cluster.local:2181"
- name: MASTER_HOST
value: "mysql-master.mysql-mgr.svc.cluster.local"
- name: SERVER_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
memory: 2Gi
volumeMounts:
- name: mysql-data
mountPath: /var/lib/mysql
- name: mysql-config
mountPath: /etc/mysql/conf.d
- name: mysql-logs
mountPath: /var/log/mysql
livenessProbe:
exec:
command:
- mysqladmin
- ping
- -h
- localhost
- -uroot
- -pRoot@2024
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
exec:
command:
- mysql
- -h
- localhost
- -uroot
- -pRoot@2024
- -e
- SELECT 1
initialDelaySeconds: 30
periodSeconds: 5
volumes:
- name: mysql-data
persistentVolumeClaim:
claimName: mysql-replica-data
- name: mysql-config
configMap:
name: mysql-config
- name: mysql-logs
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: mysql-replica-data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: fast
resources:
requests:
storage: 50Gi5. 选举控制器（Election Controller）
选举控制器负责协调主从选举，监听 ZooKeeper：
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql-election-controller
namespace: mysql-mgr
spec:
replicas: 1
selector:
matchLabels:
app: mysql-election
template:
metadata:
labels:
app: mysql-election
spec:
containers:
- name: controller
image: your-registry/mysql-election-controller:v1.0.0
imagePullPolicy: IfNotPresent
env:
- name: ZOOKEEPER_HOSTS
value: "zookeeper.mysql-mgr.svc.cluster.local:2181"
- name: MYSQL_MASTER_HOST
value: "mysql-master.mysql-mgr.svc.cluster.local"
- name: MYSQL_REPLICA_HOSTS
value: "mysql-replica-0.mysql-replica-headless.mysql-mgr.svc.cluster.local,mysql-replica-1.mysql-replica-headless.mysql-mgr.svc.cluster.local"
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mysql-secrets
key: root-password
- name: ELECTION_PATH
value: /mysql-election
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi选举控制器代码示例（Python）：
python
import kazoo
from kazoo.client import KazooClient
from kazoo.exceptions import NodeExistsException
import time
class MySQLElectionController:
def __init__(self, zk_hosts, election_path, mysql_master_host):
self.zk = KazooClient(hosts=zk_hosts)
self.election_path = election_path
self.mysql_master_host = mysql_master_host
self.is_master = False
def start(self):
self.zk.start()
self.zk.ensure_path(self.election_path)
# 创建临时顺序节点参与选举
try:
self.my_node = self.zk.create(
f"{self.election_path}/candidate_",
ephemeral=True,
sequence=True
)
except NodeExistsException:
pass
# 监听选举结果
@self.zk.ChildrenWatch(f"{self.election_path}/")
def watch_election(children):
if not children:
return
children.sort()
leader_node = f"{self.election_path}/{children[0]}"
if leader_node == self.my_node:
if not self.is_master:
self.become_master()
else:
if self.is_master:
self.become_replica()
def become_master(self):
print("Elected as master!")
self.is_master = True
# 执行变为 master 的操作
# 需要将 slave 重新配置
def become_replica(self):
print("Became replica")
self.is_master = False
def stop(self):
self.zk.stop()
# 使用示例
if __name__ == "__main__":
controller = MySQLElectionController(
zk_hosts="zookeeper.mysql-mgr.svc.cluster.local:2181",
election_path="/mysql-election",
mysql_master_host="mysql-master.mysql-mgr.svc.cluster.local"
)
try:
controller.start()
while True:
time.sleep(1)
except KeyboardInterrupt:
controller.stop()6. Secret
yaml
apiVersion: v1
kind: Secret
metadata:
name: mysql-secrets
namespace: mysql-mgr
type: Opaque
stringData:
root-password: Root@2024
repl-password: Repl@20247. StorageClass
yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd
allowVolumeExpansion: true
reclaimPolicy: Retain8. Pod 中断预算
yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: mysql-pdb
namespace: mysql-mgr
spec:
minAvailable: 1
selector:
matchLabels:
app: mysql
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: zookeeper-pdb
namespace: mysql-mgr
spec:
minAvailable: 2
selector:
matchLabels:
app: zookeeper部署步骤
1. 基础配置
bash
# 创建命名空间
kubectl apply -f 00-namespace.yaml
# 创建 StorageClass
kubectl apply -f 01-storageclass.yaml2. ZooKeeper 集群
bash
# 创建配置
kubectl apply -f 02-zk-configmap.yaml
# 创建 Service
kubectl apply -f 03-zk-services.yaml
# 创建 StatefulSet
kubectl apply -f 04-zookeeper.yaml3. MySQL 集群
bash
# 创建配置
kubectl apply -f 05-mysql-configmap.yaml
# 创建 Secret
kubectl apply -f 06-mysql-secrets.yaml
# 创建 MySQL Services
kubectl apply -f 07-mysql-services.yaml
# 创建 MySQL StatefulSets
kubectl apply -f 08-mysql-master.yaml
kubectl apply -f 09-mysql-replica.yamls4. 选举控制器
bash
kubectl apply -f 10-election-controller.yaml5. 验证
bash
# 查看 ZooKeeper 状态
kubectl exec -it zookeeper-0 -n mysql-mgr -- zookeeper-shell localhost:2181 ruok
# 查看 MySQL 主从状态
kubectl exec -it mysql-master-0 -n mysql-mgr -- mysql -uroot -pRoot@2024 -e "SHOW MASTER STATUS\G"
kubectl exec -it mysql-replica-0 -n mysql-mgr -- mysql -uroot -pRoot@2024 -e "SHOW REPLICA STATUS\G"
# 查看选举信息
kubectl logs mysql-election-controller -n mysql-mgr应用层配置示例
应用层需要配置主从感知的连接：
python
import mysql.connector
from mysql.connector import pooling
class MySQLProxy:
def __init__(self, master_host, replica_hosts, password):
self.master_host = master_host
self.replica_hosts = replica_hosts
self.password = password
# 创建连接池
self.master_pool = pooling.MySQLConnectionPool(
pool_name="master",
pool_size=5,
host=master_host,
user="appuser",
password=password,
database="appdb"
)
self.replica_pool = pooling.MySQLConnectionPool(
pool_name="replica",
pool_size=10,
host=replica_hosts[0],
user="appuser",
password=password,
database="appdb"
)
def get_write_connection(self):
"""获取写连接（到主节点）"""
return self.master_pool.get_connection()
def get_read_connection(self):
"""获取读连接（到从节点）"""
return self.replica_pool.get_connection()
def execute_on_master(self, query, params=None):
"""在主节点上执行写操作"""
conn = self.get_write_connection()
cursor = conn.cursor()
cursor.execute(query, params or ())
conn.commit()
return cursor.fetchall()
def execute_on_replica(self, query, params=None):
"""在从节点上执行读操作"""
conn = self.get_read_connection()
cursor = conn.cursor()
cursor.execute(query, params or ())
return cursor.fetchall()Spring Boot 配置：
yaml
spring:
datasource:
url: jdbc:mysql:loadbalance://mysql-replica.mysql-mgr.svc.cluster.local:3306/appdb
username: appuser
password: App@2024
hikari:
maximum-pool-size: 20
minimum-idle: 5故障转移测试
1. 模拟主节点故障
bash
# 删除主节点 Pod
kubectl delete pod mysql-master-0 -n mysql-mgr2. 观察选举变化
bash
# 查看选举控制器日志
kubectl logs mysql-election-controller -n mysql-mgr -f
# 查看主节点变化
kubectl get pods -n mysql-mgr -l app=mysql -o wide3. 验证服务恢复
bash
# 测试写入
kubectl exec -it mysql-replica-0 -n mysql-mgr -- \
mysql -uroot -pRoot@2024 -e "INSERT INTO test.t VALUES(1, NOW())"
# 验证数据同步
kubectl exec -it mysql-replica-1 -n mysql-mgr -- \
mysql -uroot -pRoot@2024 -e "SELECT * FROM test.t"常见问题排查
ZooKeeper 无法选举
检查 ZooKeeper 节点数是否达到多数（>= 3）；检查网络连通性。
MySQL 复制中断
检查 SHOW REPLICA STATUS 错误信息；验证 binlog 格式是否一致。
选举控制器异常
检查 ZooKeeper 连接配置；验证选举路径权限。
总结
本文档提供了使用 ZooKeeper 进行主从选举的 MySQL 高可用部署方案。关键点包括：
- ZooKeeper 集群：3 节点保证高可用
- MySQL 主从：1 主 2 从架构
- 选举控制器：监听 ZooKeeper，协调主从切换
- 应用层感知：需要配置读写分离
真正的故障自动转移还需要：
- 配合服务注册发现（如 Consul）
- 应用层连接池支持故障转移
- 完整的监控告警系统