k8s-部署单master节点

---

前言

实验环境规划：
操作系统：全新的rocky9.6
配置： 4核cpu +/8Gib 内存/100G磁盘
网卡模式：NAT

K8S集群角色	IP	主机名
控制节点master	192.168.168.145	K8s-1
工作节点node1	192.168.168.146	K8s-2
工作节点node2	192.168.168.147	K8s-3

不做特殊说明所有机器都需要做相同的操作，只不过这里我使用k8s1实验部分脚本是否可行，k8s2来验证每一步是否可行，k8s3实验所有脚本是否可行

---

一、环境初始化

1.1配置主机名

[root@k8s-1 ~]# hostnamectl set-hostname k8s-1
[root@k8s-2 ~]# hostnamectl set-hostname k8s-2
[root@k8s-3 ~]# hostnamectl set-hostname k8s-3

1.2修改机器静态IP

[root@k8s-2 ~]# vim /etc/NetworkManager/system-connections/ens33.nmconnection
[ipv4]
address1=192.168.168.146/24
dns=114.114.114.114
gateway=192.168.168.2
method=manual

修改好配置文件之后要刷新服务之后才能生效

[root@k8s-2 ~]# nmcli c down ens33 && nmcli c up ens33

不成功可先执行

[root@k8s-2 ~]# nmcli c reload

再用新的ip重连

1.3 关闭selinux和firewalld

临时关闭selinux
setenforce 0
查看是否修改成功
getenfoce
永久修改selinux
sed -i ‘/^SELINUX=/ s/enforcing/disabled/’ /etc/selinux/config
关闭防火墙
systemctl stop firewalld
systemctl disable firewalld

1.4 配置主机hosts文件（可选）

方便相互之间通过主机名互相访问
修改每台机器的/etc/hosts文件，增加如下三行：

192.168.168.145 k8s-1
192.168.168.146 k8s-2
192.168.168.147 k8s-3

1.5 配置主机之间的免密通道（可选）

[root@k8s-2 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:mr5x7gGA90WEFrfL9f7vXuK5VsRkWGCU1ywUHpNRnbY root@k8s-2
The key's randomart image is:
+---[RSA 3072]----+
|      .++    oX@*|
|   .  oo .   o==B|
|  . o.  o .   o*.|
|   . o o o .   Eo|
|      o S   .  . |
|       +   .    .|
|      + o   . ...|
|     . + .   o.o.|
|      ooo    .==+|
+----[SHA256]-----+[root@k8s-2 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub k8s-1
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'k8s-1 (192.168.168.145)' can't be established.
ED25519 key fingerprint is SHA256:MG3kgRXVf//jIktPa5ysLPziy6VkB26VksYBU6ve4lo.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@k8s-1's password: Number of key(s) added: 1Now try logging into the machine, with:   "ssh 'k8s-1'"
and check to make sure that only the key(s) you wanted were added.

测试

[root@k8s-2 ~]# ssh k8s-1
Last login: Mon Sep 15 11:47:51 2025 from 192.168.168.1
[root@k8s-1 ~]#

1.6 关闭交换分区swap，提升性能

临时关闭

[root@k8s2 ~]# swapoff -a

永久关闭

[root@k8s2 ~]# sed -i ‘/swap/s/^/#/’ /etc/fstab

查看swap分区是否关闭

[root@k8s-1 ~]# free
total used free shared buff/cache available
Mem: 7832716 1466168 5314304 12424 1318736 6366548
Swap: 0 0 0

为什么要关闭swap交换分区？
swap是交换分区，如果机器内存不够，会使用swap分区，但是swap分区的性能较低，k8s设计的时候为了能提升性能，默认是不允许使用交换分区的
Kubeadm初始化的时候会检测swap是否关闭，如果没关闭，那就初始化失败。如果不想要关闭交换分区，安装k8s的时候可以指定–ignore-preflight-errors=Swap来解决

1.7 修改机器内核参数

[root@k8s2 ~]# cat <<EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
[root@k8s2 ~]# modprobe overlay
[root@k8s2 ~]# modprobe br_netfilter
[root@k8s2 ~]# cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
[root@k8s2 ~]# sysctl -p /etc/sysctl.d/k8s.conf

查看

[root@k8s2 ~]# lsmod | egrep “br_netfilter|overlay”
br_netfilter 36864 0
bridge 417792 1 br_netfilter
overlay 229376 14

1.8 添加防火墙模块到内核里

考虑开机能启动，也可放到/etc/rc.local

[root@k8s-2 ~]#cat > /etc/modules-load.d/ipvs.modules <<EOF
#!/bin/bash
modprobe – ip_vs
modprobe – ip_vs_rr
modprobe – ip_vs_wrr
modprobe – ip_vs_sh
modprobe – nf_conntrack
EOF

为脚本文件添加执行权限

[root@k8s-2 ~]#chmod +x /etc/modules-load.d/ipvs.modules

执行脚本文件

[root@k8s-2 ~]#/bin/bash /etc/modules-load.d/ipvs.modules

验证模块是否加载成功

[root@k8s-2 ~]# lsmod | grep -e ip_vs -e nf_conntrack
ip_vs_sh 12288 0
ip_vs_wrr 12288 0
ip_vs_rr 12288 0
ip_vs 237568 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack 229376 3 nf_nat,nft_ct,ip_vs
nf_defrag_ipv6 24576 2 nf_conntrack,ip_vs
nf_defrag_ipv4 12288 1 nf_conntrack
libcrc32c 12288 5 nf_conntrack,nf_nat,nf_tables,xfs,ip_vs

---

二、安装docker服务

2.1 配置阿里云docker的repo源和国内镜像加速器

[root@k8s-2 ~]# yum install -y yum-utils
[root@k8s-2 ~]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -y

2.2 下载 docker

注意：不要下最新版的！！！！从这里开始可以做一个快照

k8s1.24版本后，默认的容器运行时软件变成了containerd，但是还是可以支持docker，需要额外安装一个软件

[root@k8s-2 ~]# yum install -y docker-ce-20.10.24 docker-ce-cli-20.10.24 containerd.io-1.6.20

当你不指定具体的发行版后缀（如 .el7/.el9）时，DNF/YUM 会根据当前系统的版本（Rocky Linux 9 对应 .el9）自动匹配仓库中最合适的包版本，从而避免版本后缀不匹配的问题

2.3 启动并设置开机自启

[root@k8s-2 ~]# systemctl start docker
[root@k8s-2 ~]# systemctl enable docker

查看docker进程

[root@k8s-2 ~]# ps aux|grep docker
root 1014 1.2 1.3 1759656 105880 ? Ssl 09:21 1:32 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
root 146230 0.0 0.0 6408 2176 pts/0 S+ 11:27 0:00 grep --color=auto docker

2.4 配置镜像加速器

[root@k8s2 ~]# mkdir -p /etc/docker
[root@k8s-2 docker]# tee /etc/docker/daemon.json <<EOF
{
“registry-mirrors”:[“https://docker.xuanyuan.me”,“https://docker.1panel.live”], “exec-opts”: [“native.cgroupdriver=systemd”]
}
EOF
[root@k8s2 ~]# systemctl daemon-reload
[root@k8s-2 docker]# systemctl restart docker

---

三、部署 kubernetes

3.1 配置安装k8s组件需要的阿里云的repo源

[root@k8s2 ~]# cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

3.2 安装kubeadm、kubelet、kubectl

[root@k8s2 ~]# yum install -y kubeadm-1.23.17-0 kubelet-1.23.17-0 kubectl-1.23.17-0 --disableexcludes=kubernetes
[root@k8s2 ]# kubelet --version
Kubernetes v1.23.17
[root@k8s2 ~]# systemctl start kubelet
[root@k8s2 ~]# systemctl enable kubelet

• kubeadm: kubeadm是一个工具，用来初始化k8s集群的 --》安装k8s的
• kubelet: 安装在集群的所有节点上，用于启动Pod的 --》启动pod的，管理docker的
• kubectl: 通过kubectl可以部署和管理应用，查看各种资源，创建、删除和更新各种组件 --》命令行工具

还有二进制安装，要去下载所有组件的二进制程序，在多台宿主机上安装
下载地址：https://github.com/kubernetes/kubernetes/releases

3.3 初始化 kubernetes

确保一下端口未被占用：

• 6443：API Server
• 2370：etcd
• 10250：Kubelet
• 10251：kube-scheduler
• 10252：kube-controller-manager

只在master节点上做，初始化kubernetes集群

[root@k8s1 ~]# kubeadm init \--kubernetes-version=v1.23.17 \--pod-network-cidr=10.224.0.0/16 \--service-cidr=10.96.0.0/12 \--apiserver-advertise-address=192.168.168.145 \
--image-repository=registry.aliyuncs.com/google_containers

• 第一行指定kubernetes版本为1.23.17
• 第二行指定pod网络的CIDR范围，需要与后续安装的网络插件配置一直
• 第三行指定service的CIDR范围
• 第四行指定API Server的地址，也是其他机器节点访问API Server的地址
• 第五行指定镜像仓库地址

命令后面要用

查看kubelet是否开启

设置开机自启

[root@k8s-1 ~]# systemctl enable kubelet

配置 kubectl 客户端

[root@k8s1 ~]# mkdir -p $HOME/.kube
[root@k8s1 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s1 ~]# chown $(id -u):$(id -g) $HOME/.kube/config

验证 kube-apiserver 端口

[root@k8s1 ~]# ss -anplut|grep 6443
tcp    LISTEN     0      128    [::]:6443               [::]:*                   users:(("kube-apiserver",pid=2234,fd=7))

3.4 node节点加入集群

[root@k8s2 ~]# kubeadm join 192.168.168.145:6443 --token 70zfsp.f8wiwxbnwx3u6lac \--discovery-token-ca-cert-hash sha256:1e4f5c7e7af7d5b502c49b26e50bfef2a1990207a10545aa22f6880ce316ef58 

如果不记得命令，可以在master节点上查找

[root@k8s-1 pod]#  kubeadm token create --print-join-command
kubeadm join 192.168.168.145:6443 --token 7ert39.xjntgpzy43g0x925 --discovery-token-ca-cert-hash sha256:1e4f5c7e7af7d5b502c49b26e50bfef2a1990207a10545aa22f6880ce316ef58 

检查node状态

[root@k8s-1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-1 NotReady control-plane,master 100m v1.23.17
k8s-2 NotReady 16m v1.23.17

设置k8s-2的角色为worker 后面记得补k8s3!!!

[root@k8s1 ~]# kubectl label node k8s-2 node-role.kubernetes.io/worker=worker
node/k8s-2 labeled
[root@k8s1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s1 NotReady control-plane,master 100m v1.23.17
k8s2 NotReady worker 16m v1.23.17

3.5 安装calico网络插件

calico是k8s集群里网络通信的一个组件（软件），实现集群内部不同的机器之间的容器的通信（大的网络规模）。k8s集群节点可以到5000节点，1亿个容器

flannel 也是一个网络插件（适合小规模的集群，节点服务器的数量比较小，例如：10台左右）
terway 是阿里云自己研发的一个网络插件

[root@k8s-1 ~]# kubectl apply -f https://docs.projectcalico.org/archive/v3.25/manifests/calico.yaml
[root@k8s-1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s1 NotReady control-plane,master 106m v1.23.17
k8s2 Ready worker 21m v1.23.17

过一会都是Ready
查看docker镜像，会有一个calico镜像

[root@k8s-1 ~]# docker images

可能网络不好拉不下来镜像，启动不起来

解决方案

• 导入calico的镜像 -> 每个节点都要做

[root@k8s-1 ~]# ls
anaconda-ks.cfg calico.cni.tar calico.node.tar coredns.tar
[root@k8s-1 ~]# docker load -i calico.cni.tar
[root@k8s-1 ~]# docker load -i calico.node.tar
[root@k8s-1 ~]# docker load -i coredns.tar

清理之前的 Calico 残留资源

[root@k8s-1 ~]# kubectl delete -f calico.yaml

重新部署 Calico

[root@k8s-1 ~]# kubectl apply -f calico.yaml
[root@k8s-1 ~]# kubectl get pods -n kube-system

3.6 k8s配置ipvs

在44行加ipvs，然后退出

[root@k8s1 ~]# kubectl edit configmap kube-proxy -n kube-system
configmap/kube-proxy edited

除kube-proxy Pod 是为了让配置的 IPVS 模式生效，因为kube-proxy的运行模式无法动态切换，必须通过重启（删除 Pod 触发重建）才能应用新配置

[root@k8s1 ~]# kubectl delete pods -n kube-system -l k8s-app=kube-proxy
pod “kube-proxy-5t569” deleted
pod “kube-proxy-zw9jl” deleted

3.7 配置node节点上可以使用kubectl

[root@k8s-2 ~]# kubectl get node
The connection to the server localhost:8080 was refused - did you specify the right host or port?[root@k8s-2 ~]# mkdir -p $HOME/.kube
You have new mail in /var/spool/mail/root
[root@k8s-2 ~]# scp  k8s-1:/etc/kubernetes/admin.conf /root/.kube/config
[root@k8s-2 ~]#  chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s-2 ~]#  kubectl get node
NAME   STATUS   ROLES                  AGE    VERSION
k8s-1   Ready    control-plane,master   166m   v1.23.17
k8s-2   Ready    worker                 81m    v1.23.17

3.8 安装dashboard

[root@k8s-1 ~]# yum install wget -y
[root@k8s-1 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
[root@k8s-1 ~]# kubectl apply -f recommended.yaml

查看

[root@k8s-1 ~]# kubectl get pods,svc -n kubernetes-dashboard
NAME                                             READY   STATUS    RESTARTS   AGE
pod/dashboard-metrics-scraper-6f669b9c9b-clkmx   1/1     Running   0          4m42s
pod/kubernetes-dashboard-758765f476-zwjf6        1/1     Running   0          4m43sNAME                                TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
service/dashboard-metrics-scraper   ClusterIP   10.97.251.89   <none>        8000/TCP   4m44s
service/kubernetes-dashboard        ClusterIP   10.109.54.60   <none>        443/TCP    4m48s

修改Service部分，改为NodePort对外暴露端口

[root@k8s-1 ~]# vim recommended.yaml

解决token默认15分钟过期的问题

193       containers:
194         - name: kubernetes-dashboard
195           image: kubernetesui/dashboard:v2.7.0
196           imagePullPolicy: Always
197           ports:
198             - containerPort: 8443
199               protocol: TCP
200           args:
201             - --auto-generate-certificates
202             - --namespace=kubernetes-dashboard
203             - --token-ttl=43200         #添加这条配置，超时时间调整为12小时

重新应用

[root@k8s-1 ~]# kubectl apply -f recommended.yaml

创建账号

[root@k8s-1 ~]# vim dashboard-access-token.yaml 
# Creating a Service Account
apiVersion: v1
kind: ServiceAccount
metadata:name: admin-usernamespace: kubernetes-dashboard
---
# Creating a ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: admin-user
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: cluster-admin
subjects:
- kind: ServiceAccountname: admin-usernamespace: kubernetes-dashboard
---
# Getting a long-lived Bearer Token for ServiceAccount
apiVersion: v1
kind: Secret
metadata:name: admin-usernamespace: kubernetes-dashboardannotations:kubernetes.io/service-account.name: "admin-user"
type: kubernetes.io/service-account-token
# Clean up and next steps
# kubectl -n kubernetes-dashboard delete serviceaccount admin-user
# kubectl -n kubernetes-dashboard delete clusterrolebinding admin-user

执行

[root@k8s-1 ~]# kubectl apply -f dashboard-access-token.yaml
serviceaccount/admin-user created
clusterrolebinding.rbac.authorization.k8s.io/admin-user created
secret/admin-user created

获取token

[root@k8s-1 ~]# kubectl get secret admin-user -n kubernetes-dashboard -o jsonpath={“.data.token”} | base64 -d

获取端口

[root@k8s-1 ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.97.251.89 8000/TCP 14m
kubernetes-dashboard NodePort 10.109.54.60 443:30088/TCP 14m

浏览器访问30088端口，输入获取的token即可进入

---

安装脚本

脚本说明：
$1为主机名
$2为要配置的静态IP地址
其他定义的变量如ip可在脚本中修改
脚本可跑到安装部署kubernetes

脚本中修改网络配置后，并没有刷新，防止中断脚本
init_env.sh

#!/bin/bashset -e
yum install vim psmisc tree net-tools wget -y#定义变量
gateway="192.168.168.2"
k8s1="192.168.168.145"
k8s2="192.168.168.146"
k8s3="192.168.168.147"#修改主机名
hostnamectl set-hostname $1#关闭selinux和firewalld
setenforce 0
sed -i '/^SELINUX=/ s/enforcing/disabled/' /etc/selinux/config
systemctl stop firewalld
systemctl disable firewalld#配置静态ip
sed -i '/^\[ipv4\]$/a address1='"$2"'/24\ngateway='"$gateway"'\ndns=114.114.114.114' "/etc/NetworkManager/system-connections/ens33.nmconnection" 
sed -i '/^\[ipv4\]$/,/^\[/{0,/method=auto/s//method=manual/;}' "/etc/NetworkManager/system-connections/ens33.nmconnection"#建立集群通信
cat >>/etc/hosts <<EOF
$k8s1 k8s-1
$k8s2 k8s-2
$k8s3 k8s-3
EOF#关闭交换分区swap
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab#修改机器内核参数
cat <<EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF# 加载网桥过滤模块
modprobe overlay
modprobe br_netfilter# 编辑/etc/sysctl.d/kubernetes.conf文件，主要是对容器虚拟网络的支持，添加如下配置:
echo "modprobe br_netfilter" >> /etc/profile
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl -p /etc/sysctl.d/k8s.conf# 配置防火墙模块  modprobe加载模块到内核中
cat <<EOF > /etc/modules-load.d/ipvs.modules
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF# 为脚本文件添加执行权限
chmod +x /etc/modules-load.d/ipvs.modules
# 执行脚本文件
/bin/bash /etc/modules-load.d/ipvs.modules#配置时间同步
systemctl restart chronyd && systemctl enable chronyd#安装docker
yum install -y yum-utils
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -y
yum install -y docker-ce-20.10.24 docker-ce-cli-20.10.24 containerd.io-1.6.20
systemctl start docker
systemctl enable docker
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<EOF
{"registry-mirrors":["https://docker.xuanyuan.me","https://docker.1panel.live"],                                       "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
systemctl daemon-reload
systemctl restart docker#安装k8s组件
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
# 安装kubelet、kubeadm、kubectl
yum install -y kubeadm-1.23.17-0 kubelet-1.23.17-0 kubectl-1.23.17-0 --disableexcludes=kubernetes
systemctl start kubelet
systemctl enable kubelet

[root@k8s1 ~]# bash init_env.sh k8s-3 192.168.168.147