使用k8s实现部署MySQL的主从复制
部署说明
my.cnf 配置文件抽取为 ConfigMap
将 conf、logs、data 等数据挂载到宿主机上,在 k8s 上将配置文件抽离出来做成 ConfigMap,主从节 点使用不同的配置。
MySQL 初始化时使用的密码信息抽取为 Secret
MySQL 部署初始化时可以提供 root 用户的密码,本方案使用 Secret,且主从节点使用相同的配置。
使用 StatefulSet 创建 MySQL 服务
部署 MySQL 服务使用 StatefulSet 而不选择 Deployment。
使用 PVC 持久化数据
使用 StatefulSet 的 volumeClaimTemplates 功能动态创建 PVC 存储卷声明。
集群内访问使用 DNS 提供稳定的域名
主从节点分别创建 Headless Service 服务,主从同步时使用 Headless Service 服务的 DNS 域名。
应用访问
k8s 集群内的应用,访问 MySQL 主节点的 Headless Service 对应的 DNS 域名,集群外的应用通过 mysql-source-external 服务的 NodePort 端口访问 MySQL 主节点。
创建步骤
一、创建ConfigMap
1.1主节点的ConfigMap配置
使用 vi 编辑器,新建从节点 my.cnf 资源清单文件 mysql-source-cnf.yaml ,并输入以下内容:
kind: ConfigMapapiVersion: v1metadata:name: mysql-source-cnfdata:mysqld.cnf: |
[mysqld]# performance settingslock_wait_timeout = 3600open_files_limit = 65535back_log = 1024max_connections = 2048max_connect_errors = 1000000table_open_cache = 1024table_definition_cache = 1024thread_stack = 512Ksort_buffer_size = 4Mjoin_buffer_size = 4Mread_buffer_size = 8Mread_rnd_buffer_size = 4Mbulk_insert_buffer_size = 64Mthread_cache_size = 768interactive_timeout = 600wait_timeout = 600tmp_table_size = 32Mmax_heap_table_size = 32Minnodb_open_files = 1024# Replication settingsserver_id = 1log_bin = mysql-binlog_bin_index = mysql-bin.indexbinlog_format = row
1.2从节点的ConfigMap配置
使用 vi 编辑器,新建从节点 my.cnf 资源清单文件 mysql-replica-cnf.yaml ,并输入以下内容:
kind: ConfigMapapiVersion: v1metadata:name: mysql-replica-cnf
namespace: default#annotations:# kubesphere.io/creator: admindata:mysqld.cnf: |
[mysqld]# performance settingslock_wait_timeout = 3600open_files_limit = 65535back_log = 1024max_connections = 2048max_connect_errors = 1000000table_open_cache = 1024table_definition_cache = 1024thread_stack = 512Ksort_buffer_size = 4Mjoin_buffer_size = 4Mread_buffer_size = 8Mread_rnd_buffer_size = 4Mbulk_insert_buffer_size = 64Mthread_cache_size = 768interactive_timeout = 600wait_timeout = 600tmp_table_size = 32Mmax_heap_table_size = 32Minnodb_open_files = 1024# TLS 配置
tls_version = TLSv1.2# Replication settingsserver_id = 2log_bin = mysql-binlog_bin_index = mysql-bin.indexbinlog_format = rowrelay-log = mysql-relay-binrelay-log-index = mysql-relay-bin.indexskip_slave_start = 1log_slave_updates = 1read_only = 1
1.3创建资源
创建ConfigMap资源
[root@k8s-master01 3]# kubectl apply -f mysql-replica-cnf.yaml -f mysql-source
cnf.yamlconfigmap/mysql-replica-cnf createdconfigmap/mysql-source-cnf created
二、创建Secret
2.1 Secret配置
创建一个 Secret 用来存储 MySQL root 用户的密码,主从使用相同的 Secret 配置。
运行 echo -n "123456!" | base64 命令生成 base64 编码的密码。
使用 vi 编辑器,新建 MySQL Secret 资源清单文件 mysql-secret.yaml ,并输入以下内容:
kind: SecretapiVersion: v1metadata:name: mysql-secretdata:MYSQL_ROOT_PASSWORD: MTIzNDU2type: Opaque
2.2 创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-secret.yaml
secret/mysql-secret created
三、创建Service
3.1 创建主节点headless服务
使用 vi 编辑器,新建 MySQL 主节点 headless 资源清单文件 mysql-source-svc.yaml ,并输入以下内容:
kind: ServiceapiVersion: v1metadata:name: mysql-source-headlesslabels:app: mysql-sourcespec:
ports:- name: tcp-3306protocol: TCPport: 3306targetPort: 3306selector:app: mysql-sourceclusterIP: Nonetype: ClusterIP
再执行下面的命令创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-replica-svc.yaml
service/mysql-replica-headless created
3.2 创建从节点的headless服务
使用 vi 编辑器,新建 MySQL 从节点 headless 资源清单文件 mysql-replica-svc.yaml ,并输入以 下内容:
kind: ServiceapiVersion: v1metadata:name: mysql-replica-headlesslabels:app: mysql-replicaspec:ports:- name: tcp-3306protocol: TCPport: 3306targetPort: 3306selector:app: mysql-replicaclusterIP: Nonetype: ClusterIP
再执行下面命令,创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-replica-svc.yaml
service/mysql-replica-headless created
3.3 创建外部访问服务
使用vi编辑器,新建MySQL从节点headless资源清单文件mysql-external-svc.yaml,并输入以下内容:
apiVersion: v1kind: Servicemetadata:name: mysql-source-externalspec:type: NodePortselector:app: mysql-sourceports:- port: 3306targetPort: 3306nodePort: 31306
再执行下面命令,创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-external-svc.yaml
service/mysql-source-external created
四、创建stateSet有状态服务
在 Kubernetes 集群中部署数据库服务时,我们面临着选择有状态服务(StatefulSet)与无状态服务 (Deployment)之间的决策。对于 MySQL 这类数据库服务,我们选择使用 StatefulSet 而不是 Deployment,原因如下:
稳定的网络身份:StatefulSet 为每个 Pod 分配了一个持久且唯一的网络标识符,这对于 MySQL 这 类需要固定主机名或网络地址以维持主从复制关系的数据库服务至关重要。
持久化存储:StatefulSet 易于与持久化存储卷结合使用,确保数据库数据的持久保存,即便是在 Pod 重启或重新调度后。
适合有状态应用:StatefulSet 是为有状态应用设计的,如数据库和消息队列,它提供了必要的支持 来维护这些应用的状态。
4.1 创建主节点StatefulSet
使用 vi 编辑器,新建 MySQL 主节点 StatefulSet 资源清单文件 mysql-source-sts.yaml ,并输入以 下内容:
kind: StatefulSet
apiVersion: apps/v1metadata:name: mysql-sourcelabels:app: mysql-sourcespec:replicas: 1selector:matchLabels:app: mysql-sourcetemplate:metadata:labels:app: mysql-sourcespec:volumes:- name: host-timehostPath:path: /etc/localtimetype: ''- name: configconfigMap:name: mysql-source-cnfdefaultMode: 420containers:- name: mysql-sourceimage: mysql:8.4.3imagePullPolicy: IfNotPresentports:- containerPort: 3306env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: MYSQL_ROOT_PASSWORDresources:limits:cpu: '2'memory: 4Girequests:cpu: 100mmemory: 100MivolumeMounts:- name: host-timemountPath: /etc/localtime- name: datamountPath: /var/lib/mysql- name: configreadOnly: truemountPath: /etc/mysql/conf.d/volumeClaimTemplates:- metadata:name: dataspec:accessModes: ["ReadWriteOnce"]resources:requests:storage: 10GistorageClassName: nfs-clientvolumeMode: FilesystemserviceName: mysql-source-headless
再创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-source-sts.yaml
statefulset.apps/mysql-source created
4.2创建从节点StatefulSet
使用 vi 编辑器,新建 MySQL 从节点 StatefulSet 资源清单文件 mysql-replica-sts.yaml,并输入 以下内容:
kind: StatefulSetapiVersion: apps/v1metadata:name: mysql-replicalabels:app: mysql-replicaspec:replicas: 1selector:matchLabels:app: mysql-replicatemplate:metadata:labels:app: mysql-replicaspec:volumes:- name: host-timehostPath:path: /etc/localtimetype: ''- name: configconfigMap:name: mysql-replica-cnfdefaultMode: 420containers:- name: mysql-replicaimage: mysql:8.4.3imagePullPolicy: IfNotPresentports:- containerPort: 3306env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: MYSQL_ROOT_PASSWORDresources:limits:cpu: '2'memory: 4Girequests:cpu: 100mmemory: 100MivolumeMounts:- name: host-timemountPath: /etc/localtime- name: datamountPath: /var/lib/mysql- name: configreadOnly: truemountPath: /etc/mysql/conf.d/volumeClaimTemplates:- metadata:name: dataspec:accessModes: ["ReadWriteOnce"]resources:requests:storage: 10GistorageClassName: nfs-clientvolumeMode: FilesystemserviceName: mysql-replica-headless
再创建资源
[root@k8s-master01 3]# kubectl apply -f mysql-replica-sts.yaml
statefulset.apps/mysql-replica created
4.3 验证MySQL主从Pod的状态
查看Pod状态
[root@k8s-master01 3]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE
NOMINATED NODE READINESS GATESmysql-replica-0 1/1 Running 0 13m 10.244.58.243 k8s-node02
<none> <none>mysql-source-0 1/1 Running 0 13m 10.244.58.246 k8s-node02
<none> <none>
验证自定义配置是否生效
[root@k8s-master01 3]# kubectl exec -it mysql-source-0 -- mysql -uroot -p123456
e "show variables like '%max_conn%';"
五、 配置主从同步
5.1主节点配置
5.1.1进入MySQL主节点容器内部
[root@k8s-master01 3]# kubectl exec -it mysql-source-0 -- mysql -uroot -pEnter password:
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 9Server version: 8.4.3 MySQL Community Server - GPLCopyright (c) 2000, 2024, Oracle and/or its affiliates.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>
5.1.2 创建主从同步用户
-- 创建用户并设置密码
mysql> CREATE USER 'repuser'@'%' IDENTIFIED BY '123456';Query OK, 0 rows affected (0.01 sec)
-- 赋予权限
mysql> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'repuser'@'%';Query OK, 0 rows affected (0.01 sec)
-- 刷新权限
mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)mysql> create user 'repuser'@'%' identified by '123456';Query OK, 0 rows affected (0.03 sec)mysql> grant replication slave,replication client on *.* to 'repuser'@'%';Query OK, 0 rows affected (0.00 sec)mysql> flush privileges;Query OK, 0 rows affected (0.00 sec)mysql>
5.1.3 查看 Master 状态
mysql> SHOW BINARY LOG STATUS;+------------------+----------+--------------+------------------+-------------------+| File
| Position | Binlog_Do_DB | Binlog_Ignore_DB |
Executed_Gtid_Set |+------------------+----------+--------------+------------------+-------------------+| mysql-bin.000003 |
884 |
mysql>
|
|
|+------------------+----------+--------------+------------------+-------------------+1 row in set (0.00 sec)
5.2 从节点
5.2.1 进入MySQL从节点容器内部
[root@k8s-master01 3]# kubectl exec -it mysql-replica-0 -- mysql -uroot -pEnter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 8Server version: 8.4.3 MySQL Community Server - GPLCopyright (c) 2000, 2024, Oracle and/or its affiliates.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>
5.2.2 配置主从同步
mysql> CHANGE REPLICATION SOURCE TO SOURCE_HOST='mysql-source-headless.default.svc.cluster.local',SOURCE_PORT=3306,SOURCE_USER='repuser',SOURCE_PASSWORD='123456',SOURCE_LOG_FILE='mysql-bin.000003',SOURCE_LOG_POS=884;Query OK, 0 rows affected, 2 warnings (0.03 sec)
5.2.3启动主从同步
mysql> start replica;Query OK, 0 rows affected (0.01 sec)
5.2.4 查看主从同步状态
mysql> show replica status \G;*************************** 1. row ***************************Slave_IO_State: Waiting for master to send eventMaster_Host: mysql-source-headless.default.svc.cluster.localMaster_User: repuserMaster_Port: 3306Connect_Retry: 60Master_Log_File: binlog.000003Read_Master_Log_Pos: 884Relay_Log_File: mysql-relay-bin.000005Relay_Log_Pos: 986Relay_Master_Log_File: mysql-bin.000003Slave_IO_Running: Yes # 重点注意Slave_SQL_Running: Yes # 重点注意Replicate_Do_DB:Replicate_Ignore_DB:Replicate_Do_Table:Replicate_Ignore_Table:Replicate_Wild_Do_Table:Replicate_Wild_Ignore_Table:Last_Errno: 0Last_Error:Skip_Counter: 0Exec_Master_Log_Pos: 884Relay_Log_Space: 2949023Until_Condition: NoneUntil_Log_File:Until_Log_Pos: 0Master_SSL_Allowed: NoMaster_SSL_CA_File:Master_SSL_CA_Path:Master_SSL_Cert:Master_SSL_Cipher:Master_SSL_Key:Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: NoLast_IO_Errno: 0Last_IO_Error:Last_SQL_Errno: 0Last_SQL_Error:Replicate_Ignore_Server_Ids:Master_Server_Id: 1Master_UUID: 95818d7b-7baf-11ef-819e-1af693721830Master_Info_File: /var/lib/mysql/master.infoSQL_Delay: 0SQL_Remaining_Delay: NULLSlave_SQL_Running_State: Slave has read all relay log; waiting for more
updates # 重点注意Master_Retry_Count: 86400Master_Bind:Last_IO_Error_Timestamp:Last_SQL_Error_Timestamp:Master_SSL_Crl:Master_SSL_Crlpath:Retrieved_Gtid_Set:Executed_Gtid_Set:Auto_Position: 0Replicate_Rewrite_DB:Channel_Name:Master_TLS_Version:1 row in set (0.00 sec)