RK3588 + 银河麒麟部署 swarm 集群指南

一. 前提（所有的节点）

1）设置主机名（各台设备）

	# 在 master-node 上执行sudo hostnamectl set-hostname master-node# 在 worker-1 上执行sudo hostnamectl set-hostname worker-1# 在 worker-2 上执行sudo hostnamectl set-hostname worker-2# 验证hostname# 编辑hosts文件（所有节点）sudo vi /etc/hosts# 添加以下内容（根据实际IP修改）：192.168.137.224 master-node192.168.137.225 worker-1  # 根据实际worker-1的IP修改192.168.137.226 worker-2  # 根据实际worker-2的IP修改

2）sshd使能

	vi /etc/ssh/sshd_config修改：PermitRootLogin yessudo systemctl restart ssh

3）修改源

	vi /etc/apt/sources.list# 主仓库deb http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1 main restricted universe multiverse# 更新仓库deb http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2107-updates main restricted universe multiversedeb http://archive.kylinos.cn/kylin/KYLIN-ALL 10.1-2203-updates main restricted universe multiverse

4）更新系统 & 安装基础工具

	sudo apt update && sudo apt upgrade -ysudo apt install libcurl4=7.68.0-1kylin2.12sudo apt install -y curl wget net-tools ssh iputils-ping

5）确保时间同步

	sudo apt install -y chronysudo systemctl enable chrony --now# 验证时间同步状态chronyc sources -v		

6）关闭防火墙

	systemctl list-units | grep -E "(ufw|firewall)" //判断使用哪种# 如果使用 ufwsudo ufw disable  # 如果是 firewalldsudo systemctl stop firewalld && sudo systemctl disable firewalld

7）工作目录

	mkdir -p /home/swarm

8）modprobe欺骗

	原因：内核的版本号与kylin的文件系统的驱动版本不对应，同时内核中把某些驱动直接编译到了内核中，导致modprobe不能成功。当然了这里是swarm，k3s和k8s是必须的。# 1. 备份真实的 modprobesudo mv /sbin/modprobe /sbin/modprobe.real# 2. 创建欺骗脚本cat << 'EOF' | sudo tee /sbin/modprobe#!/bin/shMODULE="$1"if [ "$MODULE" = "br_netfilter" ] || [ "$MODULE" = "overlay" ]; thenexit 0elif [ "$MODULE" = "-r" ] || [ "$MODULE" = "--remove" ]; thenif [ "$2" = "br_netfilter" ] || [ "$2" = "overlay" ]; thenexit 0fiexec /sbin/modprobe.real "$@"elseexec /sbin/modprobe.real "$@"fiEOF# 3. 添加可执行权限sudo chmod +x /sbin/modprobe# 4. 确保 sysctl 已设置（关键！）cat << 'EOF' | sudo tee /etc/sysctl.d/99-k8s-bridge-netfilter.confnet.bridge.bridge-nf-call-iptables = 1net.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-arptables = 1EOFsudo sysctl -p# 5. 验证欺骗是否成功sudo modprobe br_netfilter && echo "✅ Success" || echo "❌ Failed"    // 应输出：✅ Successsudo modprobe overlay && echo "✅ Success" || echo "❌ Failed"

二、docker安装过程

1）安装

	sudo apt updatesudo apt install docker.iosudo usermod -aG docker $USERdocker -vDocker version 20.10.7, build 20.10.7-0kylin5~20.04.2

2）配置

	sudo tee /etc/docker/daemon.json <<-'EOF'{"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn","https://hub-mirror.c.163.com","https://mirror.baidubce.com","https://dockerproxy.com","https://mirrors.ustc.edu.cn","https://docker-cf.registry.cyou","https://dockercf.jsdelivr.fyi","https://docker.jsdelivr.fyi","https://dockertest.jsdelivr.fyi","https://mirror.aliyuncs.com","https://docker.m.daocloud.io","https://docker.nju.edu.cn","https://docker.mirrors.sjtug.sjtu.edu.cn","https://mirror.iscas.ac.cn","https://docker.rainbond.cc","https://docker.registry.cyou","https://do.nark.eu.org","https://dc.j8.work","https://gst6rzl9.mirror.aliyuncs.com","https://registry.docker-cn.com","https://mirrors.tuna.tsinghua.edu.cn","https://registry.cn-beijing.aliyuncs.com"],"insecure-registries" : ["192.168.137.224:5000","registry.docker-cn.com","http://mirrors.sohu.com","http://hub-mirror.c.163.com","docker.mirrors.ustc.edu.cn"],"debug": true,"experimental": false}EOFsystemctl daemon-reloadsystemctl restart docker.servicedocker info

3）测试

	docker system prune -f //清理之前的下载docker pull hello-worlddocker run hello-world运行结果：Hello from Docker!This message shows that your installation appears to be working correctly.To generate this message, Docker took the following steps:1. The Docker client contacted the Docker daemon.2. The Docker daemon pulled the "hello-world" image from the Docker Hub.(arm64v8)3. The Docker daemon created a new container from that image which runs theexecutable that produces the output you are currently reading.4. The Docker daemon streamed that output to the Docker client, which sent itto your terminal.To try something more ambitious, you can run an Ubuntu container with:$ docker run -it ubuntu bashShare images, automate workflows, and more with a free Docker ID:https://hub.docker.com/For more examples and ideas, visit:https://docs.docker.com/get-started/

4）docker的一些指令

# 删除不完整的镜像
docker system prune -f
# 检查镜像是否存在
docker images | grep portainer使用国内镜像源拉取 Portainer
docker pull portainer/portainer-ce:latest//镜像打包
# 检查 hello-world 镜像是否存在
docker images | grep hello-world
# 将镜像保存为 tar 文件
docker save -o hello-world.tar hello-world:latest
# 或者使用重定向方式
docker save hello-world:latest > hello-world-v2.tar
# 查看生成的文件
ls -lh hello-world*.tar//本地镜像导入
# 从 tar 文件导入镜像
docker load -i hello-world.tar
# 或者使用输入重定向
docker load < hello-world.tar
# 验证镜像已导入
docker images | grep hello-world

三、swarm安装

1）manager初始化及查看

docker swarm init --advertise-addr 192.168.137.224打印显示：Swarm initialized: current node (quj6te9i5wbhnvlukhr8umyyw) is now a manager.To add a worker to this swarm, run the following command:docker swarm join --token SWMTKN-1-09sgobht9vrzc0kzh80lvejpvyp2suda5akgmbf96dvrbiszpi-al15wyhpzlr2cyd68291mup3q 192.168.137.224:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.查看swarm集群状态: docker info查看节点信息: docker node ls打印如下：root@master-node:/home/swarm# docker node lsID                            HOSTNAME      STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSIONquj6te9i5wbhnvlukhr8umyyw *   master-node   Ready     Active         Leader           20.10.7
查询worker要用的加入指令: docker swarm join-token worker		打印如下：docker swarm join --token SWMTKN-1-09sgobht9vrzc0kzh80lvejpvyp2suda5akgmbf96dvrbiszpi-al15wyhpzlr2cyd68291mup3q 192.168.137.224:2377	

2）worker配置及查看

在worker-1和worker-2执行：docker swarm join --token SWMTKN-1-09sgobht9vrzc0kzh80lvejpvyp2suda5akgmbf96dvrbiszpi-al15wyhpzlr2cyd68291mup3q 192.168.137.224:2377打印如下：This node joined a swarm as a worker.
在manager查看节点信息: docker node ls打印如下：ID                            HOSTNAME      STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSIONquj6te9i5wbhnvlukhr8umyyw *   master-node   Ready     Active         Leader           20.10.7v7n81ykd7mvine0ul6npshbyz     worker-1      Ready     Active                          20.10.7r9zl96awq209m8kjpffr9a1es     worker-2      Ready     Active  						  20.10.7

四、部署服务hello-world

注意：：：hello-world镜像为输出一个打印，因此为执行即结束，但是集群具备故障恢复功能，因此就会一直在重复启动！！！

1）操作过程及指令

# 1. 使用已有的 hello-world 镜像创建服务
docker service create --name test-service --replicas 3 hello-world# 2. 查看服务状态
docker service ps test-service# 3. 查看服务日志
docker service logs test-service# 4. 扩展服务到更多副本
docker service scale test-service=6# 5. 再次查看分布情况
docker service ps test-service# 6. 清理测试服务
docker service rm test-service

2）日志如下

root@master-node:~# docker service ps test-service
ID             NAME                 IMAGE                NODE          DESIRED STATE   CURRENT STATE             ERROR     PORTS
anemuatoff61   test-service.1       hello-world:latest   master-node   Ready           Ready 3 seconds ago                 
0iwhwbqm1w0i    \_ test-service.1   hello-world:latest   master-node   Shutdown        Complete 3 seconds ago              
ylqqghxb9jd6    \_ test-service.1   hello-world:latest   master-node   Shutdown        Complete 9 seconds ago              
5zh3m6cf1a6n    \_ test-service.1   hello-world:latest   master-node   Shutdown        Complete 15 seconds ago             
jvec6zvc3ti0    \_ test-service.1   hello-world:latest   master-node   Shutdown        Complete 21 seconds ago             
d5j4mbjai5ay   test-service.2       hello-world:latest   worker-2      Ready           Ready 2 seconds ago                 
v6d2one9fs0c    \_ test-service.2   hello-world:latest   worker-2      Shutdown        Complete 2 seconds ago              
hbj46wuzzva2    \_ test-service.2   hello-world:latest   worker-2      Shutdown        Complete 8 seconds ago              
uq0nq6z3wh0k    \_ test-service.2   hello-world:latest   worker-2      Shutdown        Complete 15 seconds ago             
mjfrz9ie9qks    \_ test-service.2   hello-world:latest   worker-2      Shutdown        Complete 21 seconds ago             
0bti65ql81y5   test-service.3       hello-world:latest   worker-1      Ready           Ready 2 seconds ago                 
fvr1thptrn5y    \_ test-service.3   hello-world:latest   worker-1      Shutdown        Complete 3 seconds ago              
za0b02yrakt3    \_ test-service.3   hello-world:latest   worker-1      Shutdown        Complete 9 seconds ago              
rmgiomcz54g1    \_ test-service.3   hello-world:latest   worker-1      Shutdown        Complete 15 seconds ago             
i4bh61dg0o4q    \_ test-service.3   hello-world:latest   worker-1      Shutdown        Complete 21 seconds ago             
root@master-node:~# docker service logs test-service	

3）成功的证据

1. 服务调度正常3个副本分别运行在3个节点上：test-service.1 在 master-nodetest-service.2 在 worker-2test-service.3 在 worker-1
2. 负载均衡和高可用性每个任务都有多个历史记录（Shutdown → Complete → 新的 Ready），这表明 Swarm 在容器退出后自动重新调度，确保服务持续可用，这是 Swarm 自愈能力 的体现
3. 跨节点通信正常日志显示所有3个节点都成功执行了任务网络通信畅通，日志能够汇总到管理节点
4. 镜像分发正常所有节点都能访问并运行 hello-world 镜像；ARM64 架构兼容性良好；

4）🎯 集群功能验证完成

功能	状态	验证结果
服务创建	✅	成功创建多副本服务
节点调度	✅	均匀分布在3个节点
负载均衡	✅	自动分配任务
高可用性	✅	故障后自动恢复
日志收集	✅	跨节点日志汇总
网络通信	✅	节点间通信正常