使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控)
使用 Docker-Compose 部署 Redis 三主三从集群(含 Exporter 监控
- 一、环境准备
- 二、Docker Compose 配置
- 三、Redis 配置文件
- 四、ACL 权限配置
- 五、系统优化配置
- 六、启动服务
- 七、创建集群
- 八、验证集群状态
- 九、监控接入 Prometheus
- 十、常见的场景运维
- 十一、总结
本文将介绍如何在三台服务器上使用 Docker Compose 部署 Redis 三主三从集群,并配置 Redis Exporter 用于 Prometheus 监控。
部署的 Redis 版本为 7.2.4,并启用 ACL 用户权限控制与密码认证。
一、环境准备
三台服务器(示例):
主机名 IP
domp-server-001 172.20.191.185
domp-server-002 172.20.191.186
domp-server-003 172.20.191.187
┌──────────────┐ ┌──────────────┐ ┌──────────────┐│ Master #1 │ │ Master #2 │ │ Master #3 ││ 172.20.191.185:6379│ │ 172.20.191.186:6379│ │ 172.20.191.187:6379│└───────┬──────┘ └───────┬──────┘ └───────┬──────┘│ │ ││ │ │┌───────▼──────┐ ┌───────▼──────┐ ┌───────▼──────┐│ Replica #1 │ │ Replica #2 │ │ Replica #3 ││ 172.20.191.185:6380│ │ 172.20.191.186:6380│ │ 172.20.191.187:6380│└─────────────────────┘ └─────────────────────┘ └─────────────────────┘
说明:• 每个 Master 对应一个 Replica(主从配对),总共 6 个节点。• Exporter 部署在每台服务器上,分别采集 6379、6380 实例的监控数据。• Redis Cluster 负责数据分片和故障转移,节点通过 cluster bus 端口(16379/16380)通信。
Exporter:
- redis-exporter-1 (6379 metrics) → Prometheus
- redis-exporter-2 (6380 metrics) → Prometheus
软件环境:
• Docker ≥ 20.10
• Docker Compose ≥ 1.29
• Redis 镜像:reg.deeplearning.cn/mmwei3/redis:7.2.4
• Redis Exporter 镜像:reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86
目录规划:
/data/redis/├── 6379│ ├── conf│ └── data└── 6380├── conf└── data在每台服务器上执行:mkdir -vp /data/redis/6379/conf /data/redis/6379/data
mkdir -vp /data/redis/6380/conf /data/redis/6380/data
⸻
二、Docker Compose 配置
以下示例以第一台服务器 172.20.191.185 为例(其他两台服务器需替换 cluster-announce-ip 为对应 IP)。
docker-compose.yml:version: '3'services:redis-server-6379:image: reg.deeplearning.cn/mmwei3/redis:7.2.4container_name: redis-server-6379restart: unless-stoppedports:- 6379:6379- 16379:16379environment:TZ: "Asia/Shanghai"command: ["redis-server", "/6379/conf/redis.conf"]volumes:- /data/redis/6379/conf/:/6379/conf/- /data/redis/6379/data/:/6379/data/redis-server-6380:image: reg.deeplearning.cn/mmwei3/redis:7.2.4container_name: redis-server-6380restart: unless-stoppedports:- 6380:6380- 16380:16380environment:TZ: "Asia/Shanghai"command: ["redis-server", "/6380/conf/redis.conf"]volumes:- /data/redis/6380/conf/:/6380/conf/- /data/redis/6380/data/:/6380/data/redis-exporter-1:image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86restart: unless-stoppedports:- 9121:9121environment:TZ: Asia/ShanghaiREDIS_ADDR: redis://172.20.191.185:6379REDIS_USER: redis_monitorREDIS_PASSWORD: <监控用户密码已脱敏>redis-exporter-2:image: reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86restart: unless-stoppedports:- 9122:9121environment:TZ: Asia/ShanghaiREDIS_ADDR: redis://172.20.191.185:6380REDIS_USER: redis_monitorREDIS_PASSWORD: <监控用户密码已脱敏>
⸻
三、Redis 配置文件
以 6379 节点 为例(/data/redis/6379/conf/redis.conf):bind 0.0.0.0
port 6379
timeout 3600
tcp-keepalive 300
loglevel notice
databases 16# 持久化策略
save 600 100
save 120 10000
save 7200 1dir /6379/data/
masteruser clusterrepl
masterauth <复制用户密码已脱敏>aclfile /6379/conf/users.acl
replica-read-only yescluster-announce-ip "172.20.191.185"
cluster-announce-port 16379maxclients 10000
maxmemory 10gb
maxmemory-policy noevictionappendonly yes
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb# 开启集群模式
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
cluster-announce-bus-port 163796380 节点配置 /data/redis/6380/conf/redis.conf 类似,只需修改:port 6380
dir /6380/data/
cluster-announce-port 6380
cluster-config-file nodes-6380.conf
cluster-announce-bus-port 16380
⸻
四、ACL 权限配置
/data/redis/6379/conf/users.acl(所有节点相同,密码已脱敏):user default off nopass &* -@alluser root on >[Root密码] allkeys allchannels +@all
user redis_monitor on >[监控用户密码] +client +ping +info +config|get +cluster|info +slowlog +latency +memory +select +get +scan +xinfo +type +pfcount +strlen +llen +scard +zcard +hlen +xlen +eval allkeys
user clusterrepl on >[复制用户密码] +psync +replconf +ping
user appuser on >[应用用户密码] allkeys allchannels +@all -@admin -@dangerous
⸻
五、系统优化配置
# 系统参数优化
sed -i '/vm.overcommit_memory/d' /etc/sysctl.conf
echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
echo "net.core.somaxconn=2048" >> /etc/sysctl.conf
sysctl -p# 禁用透明大页
cat <<EOF >> /etc/rc.d/rc.local
if test -f /sys/kernel/mm/transparent_hugepage/enabled; thenecho never > /sys/kernel/mm/transparent_hugepage/enabled
fi
EOF
chmod +x /etc/rc.d/rc.local# 数据目录权限
chown -R polkitd:root /data/redis
⸻
六、启动服务
在每台服务器执行:
docker-compose up -d
[root@domp-server-003 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/docker-compose.yaml: `version` is obsolete
[+] Running 14/14✔ redis-server-6380 Pulled 2.9s✔ 09f376ebb190 Pull complete 1.8s✔ f89f5b53601e Pull complete 1.8s✔ 45d9097e8d82 Pull complete 1.8s✔ b14e0d774d3d Pull complete 1.9s✔ e8f49d7b64bc Pull complete 2.4s✔ 17bfefd8913f Pull complete 2.4s✔ 4f4fb700ef54 Pull complete 2.4s✔ d05f3a965cb3 Pull complete 2.4s✔ redis-exporter Pulled 1.0s✔ 4f3b0056e2b7 Pull complete 0.5s✔ 93ace162418f Pull complete 0.5s✔ cf0c47f0538a Pull complete 0.5s✔ redis-server-6379 Pulled 2.9s
[+] Running 4/4✔ Network redis_default Created 0.0s✔ Container redis-redis-exporter-1 Started 0.5s✔ Container redis-server-6380 Started 0.6s✔ Container redis-server-6379 Started 0.6s
[root@domp-server-003 redis]#
[root@domp-server-003 redis]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
57d66719bc31 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 6379/tcp, 0.0.0.0:6380->6380/tcp, :::6380->6380/tcp redis-server-6380
18ab476e8c64 reg.deeplearning.cn/mmwei3/redis:7.2.4 "docker-entrypoint.s…" 25 seconds ago Up 25 seconds 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp redis-server-6379
c8d4239de5a2 reg.deeplearning.cn/mmwei3/oliver006/redis_exporter:latest-20250723_x86 "/redis_exporter" 25 seconds ago Up 25 seconds 0.0.0.0:9121->9121/tcp, :::9121->9121/tcp redis-redis-exporter-1
747435ab6bdb reg.deeplearning.cn/mmwei3/haproxy:latest-20250725 "kolla_start" 20 hours ago Up 14 minutes haproxy
59a91a0b5223 reg.deeplearning.cn/mmwei3/keepalived:latest-20250725 "kolla_start" 29 hours ago Up 14 minutes keepalived
[root@domp-server-003 redis]#docker exec -it redis-server-6379 redis-cli \-c \-h 172.20.191.185 \-p 6379 \--user root \-a Rootpass_c2B2h15D1PfHTOEjZd \cluster nodesredis-cli -h <host> -p <port> --user root -a <password> cluster nodes[root@domp-server-001 redis]# docker compose up -d
WARN[0000] /data/docker_up/redis/docker-compose.yaml: `version` is obsolete
[+] Running 3/3✔ Container redis-redis-exporter-1 Running 0.0s✔ Container redis-server-6379 Started 0.5s✔ Container redis-server-6380 Started 0.5s
[root@domp-server-001 redis]# docker exec -it redis-server-6379 bash
root@ab02a014ade5:/data# history1 history
root@ab02a014ade5:/data# redis-cli \--user root -a Rootpass_c2B2h15D1PfHTOEjZd \--cluster create \172.20.191.185:6379 172.20.191.185:6380 \172.20.191.186:6379 172.20.191.186:6380 \172.20.191.187:6379 172.20.191.187:6380 \--cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.20.191.186:6380 to 172.20.191.185:6379
Adding replica 172.20.191.187:6380 to 172.20.191.186:6379
Adding replica 172.20.191.185:6380 to 172.20.191.187:6379
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379slots:[0-5460] (5461 slots) master
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380replicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379slots:[5461-10922] (5462 slots) master
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380replicates 119195a14a2fc9ca186ce439c0d15d587553004e
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379slots:[10923-16383] (5461 slots) master
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380replicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.20.191.185:6379)
M: 119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379slots:[0-5460] (5461 slots) master1 additional replica(s)
S: 779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380slots: (0 slots) slavereplicates 119195a14a2fc9ca186ce439c0d15d587553004e
S: 02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380slots: (0 slots) slavereplicates 2636af29cc2d6c6badf76355c923c5c2c6503a33
M: 933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379slots:[10923-16383] (5461 slots) master1 additional replica(s)
M: 2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379slots:[5461-10922] (5462 slots) master1 additional replica(s)
S: 7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380slots: (0 slots) slavereplicates 933f4247253f383476c37f4a76f3fc3b4374c6d8
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
root@ab02a014ade5:/data#
root@ab02a014ade5:/data#
⸻
七、创建集群
在其中一台服务器执行:
docker exec -it redis-server-6379 redis-cli
–user root -a <Root密码已脱敏>
–cluster create
172.20.191.185:6379 172.20.191.185:6380
172.20.191.186:6379 172.20.191.186:6380
172.20.191.187:6379 172.20.191.187:6380
–cluster-replicas 1
执行过程中会提示确认,输入 yes 即可完成集群初始化。
⸻
八、验证集群状态
docker exec -it redis-server-6379 redis-cli \--user root -a <Root密码已脱敏> \cluster infodocker exec -it redis-server-6379 redis-cli \--user root -a <Root密码已脱敏> \cluster nodes
[root@domp-server-001 redis]# cat /data/redis/6379/data/nodes-6379.conf
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380,,tls-port=0,shard-id=99b8a25667e3a4caaee705c15c9056196f417d9e slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877534397 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380,,tls-port=0,shard-id=1f2d74eabc1a9193db35317864bc5e6e0840523a slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877533000 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379,,tls-port=0,shard-id=99b8a25667e3a4caaee705c15c9056196f417d9e myself,master - 0 1753877533000 1 connected 0-5460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379,,tls-port=0,shard-id=de5219610a55a82ba3361d902ecec170a6e54dae master - 0 1753877534000 5 connected 10923-16383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379,,tls-port=0,shard-id=1f2d74eabc1a9193db35317864bc5e6e0840523a master - 0 1753877535399 3 connected 5461-10922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380,,tls-port=0,shard-id=de5219610a55a82ba3361d902ecec170a6e54dae slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877533000 5 connected
vars currentEpoch 6 lastVoteEpoch 0
[root@domp-server-001 redis]#
[root@domp-server-001 redis]# docker exec -it redis-server-6379 redis-cli \
> -c \
> -h 172.20.191.185 \
> -p 6379 \
> --user root \
> -a Rootpass_c2B2h15D1PfHTOEjZd \
> cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
779d76a726eae05c0eb2b2d1b270b9103511684a 172.20.191.186:6380@16380 slave 119195a14a2fc9ca186ce439c0d15d587553004e 0 1753877694771 1 connected
02c23926b2e01c4d89e1d12df3cfcfbb9c7bb6f7 172.20.191.187:6380@16380 slave 2636af29cc2d6c6badf76355c923c5c2c6503a33 0 1753877691764 3 connected
119195a14a2fc9ca186ce439c0d15d587553004e 172.20.191.185:6379@16379 myself,master - 0 1753877692000 1 connected 0-5460
933f4247253f383476c37f4a76f3fc3b4374c6d8 172.20.191.187:6379@16379 master - 0 1753877693769 5 connected 10923-16383
2636af29cc2d6c6badf76355c923c5c2c6503a33 172.20.191.186:6379@16379 master - 0 1753877691000 3 connected 5461-10922
7d265dba1f3baae2c6f38c72a1b30925a12ee6e1 172.20.191.185:6380@16380 slave 933f4247253f383476c37f4a76f3fc3b4374c6d8 0 1753877692766 5 connected
[root@domp-server-001 redis]#
[root@domp-server-001 redis]# docker exec -it redis-server-6379 redis-cli -c -h 172.20.191.185 -p 6379 --user root -a Rootpass_c2B2h15D1PfHTOEjZd cluster info
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:180
cluster_stats_messages_pong_sent:185
cluster_stats_messages_sent:365
cluster_stats_messages_ping_received:180
cluster_stats_messages_pong_received:180
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:365
total_cluster_links_buffer_limit_exceeded:0
[root@domp-server-001 redis]#export CNNL_LOG_LEVEL=error
⸻
九、监控接入 Prometheus
Redis Exporter 已在 Compose 中启动:• 9121 端口对应 6379• 9122 端口对应 6380Prometheus 配置示例:- job_name: 'redis-6379'static_configs:- targets: ['172.20.191.185:9121']- job_name: 'redis-6380'static_configs:- targets: ['172.20.191.185:9122']
⸻
十、常见的场景运维
1. 集群扩容 / 缩容扩容# 新节点加入集群(作为master)
redis-cli --cluster add-node 172.20.191.188:6379 172.20.191.185:6379# 分配槽位(例如新节点分配 5000 槽)
redis-cli --cluster reshard 172.20.191.185:6379 \--cluster-from all \--cluster-to <新节点ID> \--cluster-slots 5000 \--cluster-yes缩容# 将指定节点的槽位迁移给其他节点
redis-cli --cluster reshard 172.20.191.185:6379 \--cluster-from <要下线节点ID> \--cluster-to <目标节点ID> \--cluster-slots <迁移槽位数> \--cluster-yes# 删除节点
redis-cli --cluster del-node 172.20.191.185:6379 <要下线节点ID>⸻2. 主从切换与故障转移# 手动触发某个 Replica 升级为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER# 查看当前节点角色
redis-cli -h 172.20.191.185 -p 6379 ROLE⸻3. 数据迁移与重分片# 进入交互式重分片
redis-cli --cluster reshard 172.20.191.185:6379按照提示输入:1. 接收槽位的节点ID2. 迁移的槽位数量3. 来源节点ID列表(all 表示全部节点)⸻4. 密码与 ACL 管理# 设置集群访问密码(配置文件 redis.conf)
requirepass MyStrongPass# 创建监控用户,仅允许 INFO/PING
redis-cli -a MyStrongPass ACL SETUSER monitor_user on >monitor_pass +INFO +PING⸻5. 性能优化# 查看当前内存淘汰策略
redis-cli CONFIG GET maxmemory-policy# 设置淘汰策略为 allkeys-lru
redis-cli CONFIG SET maxmemory-policy allkeys-lru# 批量写入(pipeline示例)
(echo -en "SET key1 val1\r\nSET key2 val2\r\nSET key3 val3\r\n"; sleep 1) | nc 172.20.191.185 6379⸻6. 监控与告警# 查看内存使用情况
redis-cli INFO memory# 查看主从延迟
redis-cli INFO replication# 查看慢查询日志
redis-cli SLOWLOG GET 10⸻7. 升级与维护# 升级流程示例(先从库后主库)
# Replica 升级完成后手动切换为 Master
redis-cli -h 172.20.191.186 -p 6380 CLUSTER FAILOVER⸻8. 故障排查# 查看集群状态
redis-cli -h 172.20.191.185 -p 6379 CLUSTER INFO
redis-cli -h 172.20.191.185 -p 6379 CLUSTER NODES# 检查 AOF 文件完整性
redis-check-aof appendonly.aof# 检查 RDB 文件
redis-check-rdb dump.rdb⸻
十一、总结
本文介绍了如何:
• 使用 Docker Compose 部署多实例 Redis
• 配置 三主三从集群
• 启用 ACL 用户与密码认证
• 接入 Redis Exporter 进行监控