湖南手机版建站系统哪个好可口可乐营销策划方案
LVS-Linux虚拟服务器
- 一、LVS概述
- 1.1 LVS的工作方式
- 1.2 LVS的调度算法
- 1.3 LVS软件包 - ipvsadm
- 二、NAT(可做端口映射)
- 2.1 NAT工作模式
- 2.2 NAT 模式安装
- 2.2.1 配置 RS（NAT）
- 2.2.2 配置 LVS（两个网卡）
- 配置仅主机模式网卡（用于虚拟IP，便于用户访问）
- 安装 ipvsamd
- 配置NAT模式网卡（作为后端真实主机的网关）
- 2.2.3 配置客户端（仅主机）
- 2.2.4 启动ipvsadm服务（lvs）
- 2.2.5 LVS 规则匹配
- 2.2.6 功能测试
- 客户端测试
- NAT模式内核参数配置（LVS主机）
- 三、DR
- 3.1 DR工作模式
- 3.2 DR模式单网段案例
- 3.2.1 配置路由
- 3.2.2 配置RS真实服务器
- 3.2.3 配置LVS
- 3.2.4 给 RS 服务增加 VIP
- 3.2.5 在 RS 上增加内核参数（防止IP冲突）
- DR模式内核参数（RS主机）
- 3.2.6 配置 LVS 规则
- 3.2.7 测试
- 3.3 DR多网段
- 3.3.1 配置路由
- 3.3.2 配置RS真实服务器
- 3.3.3 配置LVS
- 3.3.4 增加 VIP
- 3.3.5 配置 LVS 规则
- 3.3.6 测试
- 四、Tun
- 4.1 Tun 工作模式
- 4.2 Tun 模式配置
- 4.2.1
- 4.2.2 配置RS真实服务器
- 4.2.3 配置 LVS
- 4.2.4 给 RS 增加 VIP
- 4.2.5 在 RS 上增加内核参数
- Tun模式增加内核参数（RS主机）
- 4.2.6 lvs 配置规则
- 4.2.7 测试
- 4.2.8 报错
- 五、综合案例
- 1. NFS 业务服务器（192.168.98.138）
- 2. Web服务集群（搭建RS服务器）
- 开机自启动
- 自动挂载
- 配置nginx（为了区分Web1与Web2访问的文件内容）
- 3. LVS主机（Keepalived+lvs）
- 4. 回到Web主机，修改
- 5. 客户端测试(浏览器http://192.168.98.100)
- 停止nginx服务
- 停止keepalived服务（高可用）
一、LVS概述
Keepalived是集群管理中保证集群高可用的一个服务软件，用来防止单点故障。
Linux虚拟服务器（Linux Virtual Server，LVS）是一个虚拟的服务器集群系统，用于实现 负载平衡 和 高可用性
-
优势：
- 高性能：LVS工作在内核层，性能高效，能够处理大量并发请求。
- 高可用性：通过配置Keepalived等工具，LVS可以实现高可用性，确保服务的持续运行。
- 灵活性强：支持多种负载均衡算法和工作模式，适应不同的应用场景
-
LVS架构中的术语解释：
- VS：Virtual Server ，虚拟服务
- Director： Balancer ，也叫DS(Director Server)负载均衡器、分发器
- RS：Real Server ，后端请求处理服务器，真实服务器
- CIP: Client IP ，客户端IP
- VIP：Director Virtual IP ，负载均衡器虚拟IP
- DIP：Director IP ，负载均衡器IP
- RIP：Real Server IP ，后端请求处理的服务器IP
1.1 LVS的工作方式
-
LVS-NAT（Network Address Translation）：在该模式下，负载均衡器不仅需要修改请求报文的目标地址，还需要修改响应报文的源地址，适用于小规模集群。
请求和响应都经过 lvs，它可以进行端口映射 -
LVS-DR（Direct Routing）：在该模式下，负载均衡器只修改请求报文的目标MAC地址，而不修改IP地址，后端服务器直接将响应报文发回客户端，适用于大规模集群。
请求过lvs，响应不过 lvs，lvs和后端真实服务器都拥有相同的VIP，真实服务器做响应和请求的抑制配置 -
LVS-TUN（IP Tunneling）：该模式通过 IP隧道 将请求转发到后端服务器，后端服务器直接将响应报文发回客户端，适用于地理位置分散的集群
请求过lvs，响应不过 lvs，必须开通tunl0隧道 => ipip 模式
1.2 LVS的调度算法
LVS提供了多种调度算法，以适应不同的应用需求：
- 轮询调度（Round Robin）：将请求依次分配给每个后端服务器。
- 加权轮询调度（Weighted Round Robin）：根据服务器的权重将请求分配给后端服务器。
- 最少连接调度（Least Connections）：将请求分配给当前连接数最少的后端服务器。
- 加权最少连接调度（Weighted Least Connections）：根据服务器的权重和当前连接数分配请求。
- 基于局部性的最少连接调度（Locality-Based Least Connections）：优先将请求分配给最近使用的服务器。
1.3 LVS软件包 - ipvsadm
- ipvsadm软件包安装目录
[root@lvs ~]# rpm -ql ipvsadm
/etc/sysconfig/ipvsadm-config #配置文件
/usr/lib/.build-id
/usr/lib/.build-id/0b
/usr/lib/.build-id/0b/d10d85dc0121855898c34f27a7730b50772fcc
/usr/lib/systemd/system/ipvsadm.service #服务启动文件
/usr/sbin/ipvsadm #主程序
/usr/sbin/ipvsadm-restore #规则重载工具
/usr/sbin/ipvsadm-save #规则保存工具
/usr/share/doc/ipvsadm
/usr/share/doc/ipvsadm/MAINTAINERS
/usr/share/doc/ipvsadm/README
/usr/share/man/man8/ipvsadm-restore.8.gz
/usr/share/man/man8/ipvsadm-save.8.gz
/usr/share/man/man8/ipvsadm.8.gz
- 命令使用 - ipvsadm --help
#管理集群服务规则：ipvsadm -A|E virtual-service [-s scheduler] [-p [timeout]] [-M netmask] [--pe persistence_engine] [-b sched-flags]（创建规则及指定算法）
-A：指定规则
-E：修改算法
-t:TCP协议的端口，VIP:TCP_PORT
-u:UDP协议的端口，VIP:UDP_PORT
-f:Firewall MARK,标记，一个数字
[-p [timeout]]：持久化
[-s scheduler]:指定集群的调度算法，默认为wlcipvsadm -D virtual-service #删除某一个配置信息ipvsadm -C #清空ipvsadm -R #重载，相当于ipvsadm-restoreipvsadm -S [-n] #保存，相当于ipvsadm-save#管理集群中的RSipvsadm -a|e virtual-service -r server-address [options]
-a：加真实的服务器ipvsadm -d virtual-service -r server-addressipvsadm -L|l [virtual-service] [options] #查看ipvsadm -Z [virtual-service] #清空计数器ipvsadm --set tcp tcpfin udpipvsadm --start-daemon {master|backup} [daemon-options]ipvsadm --stop-daemon {master|backup}ipvsadm -h#指定模式--gatewaying -g gatewaying (direct routing) (default)默认--ipip -i ipip encapsulation (tunneling)模式--masquerading -m masquerading (NAT)模式
二、NAT(可做端口映射)
2.1 NAT工作模式
在LVS-NAT模式下，负载均衡器通过NAT（网络地址转换）技术将客户端的请求报文目标地址修改为后端服务器的IP地址，同时记录会话信息。当后端服务器处理完请求后，负载均衡器将响应报文的源地址修改为负载均衡器的IP地址，再返回给客户端
2.2 NAT 模式安装
-
LVS服务器需要有两块网卡，一块用于虚拟IP，便于用户访问；另一块作为后端真实主机的网关。
-
在配置规则时，需要通过 -m 参数来指定 NAT 模式
-
需要在 LVS 服务器上配置ip转发内核参数
vim /etc/sysctl.conf
net.ipv4.ip_forward=1
保险起见：防止不能上外网，开启虚拟机后，先把 ipvsamd 软件安装上，再进入虚拟机进行网卡、IP等操作的修改
- 架构：
RS的网关为LVS主机的IP
节点规划：
| 主机 | 角色 | 软件 | 网络 | IP | dns |
|---|---|---|---|---|---|
| client | client | 仅主机 | 192.168.204.100/24 | ||
| lvs | lvs | ipvsadm | 仅主机 NAT | VIP192.168.204.200/24 DIP192.168.88.8/24 | |
| nginx | rs1 | nginx | NAT | 192.168.88.7/24 | 192.168.88.8 |
| nginx | rs2 | nginx | NAT | 192.168.88.17/24 | 192.168.88.8 |
2.2.1 配置 RS（NAT）
在LVS中，RS（Real Server） 是指后端真实的工作服务器，负责处理具体的业务逻辑
- 安装 nginx
dnf install nginx -y - 修改默认访问页
echo $(hostname -I) > /usr/share/nginx/html/index.html - 启动服务
systemctl start nginx - 测试
curl localhost
网关：192.168.88.8（LVS主机NAT网卡的IP-DIP）
- RS1:
[root@localhost ~]# hostnamectl hostname rs1
[root@localhost ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.88.7/24 ipv4.gateway 192.168.88.8 connection.autoconnect yes
[root@localhost ~]# nmcli c up ens160# 安装nginx
[root@rs1 ~]# dnf install nginx -y
# 修改默认访问页
[root@rs1 ~]# echo $(hostname -I) > /usr/share/nginx/html/index.html
# 启动服务
[root@rs1 ~]# systemctl start nginx
# 测试
[root@rs1 ~]# curl localhost
192.168.88.7
- RS2:
[root@localhost ~]# hostnamectl hostname rs2
[root@localhost ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.88.17/24 ipv4.gateway 192.168.88.8 connection.autoconnect yes
[root@localhost ~]# nmcli c up ens160# 安装nginx
[root@rs1 ~]# dnf install nginx -y
# 修改默认访问页
[root@rs2 ~]# echo $(hostname -I) > /usr/share/nginx/html/index.html
# 启动服务
[root@rs2 ~]# systemctl start nginx
# 测试
[root@rs2 ~]# curl localhost
192.168.88.17
2.2.2 配置 LVS（两个网卡）
LVS服务器需要有两块网卡，一块用于虚拟IP，便于用户访问；另一块作为后端真实主机的网关
开启虚拟机之前，先对 LVS 主机进行网卡添加
lvs服务器有两块网卡
第一块网卡采用仅主机模式，IP 地址为 192.168.204.200
第二块网卡采用 NAT 模式，IP 地址为192.168.88.8
# 查看网络设备连接名称：
[root@localhost ~]# nmcli c show
NAME UUID TYPE DEVICE
Wired connection 1 716944d5-683d-3297-9f0b-39530d65c935 ethernet ens224
ens160 c6adadcb-89ef-3ed8-a265-09260f58abe1 ethernet ens160
lo 3919112a-6bb6-4ad3-a213-3ce8394ca9a9 loopback lo
# 修改连接名称
[root@localhost ~]# nmcli c modify 'Wired connection 1' connection.id ens224
# 查看修改结果
[root@localhost ~]# nmcli c show
NAME UUID TYPE DEVICE
ens224 716944d5-683d-3297-9f0b-39530d65c935 ethernet ens224
ens160 c6adadcb-89ef-3ed8-a265-09260f58abe1 ethernet ens160
lo 3919112a-6bb6-4ad3-a213-3ce8394ca9a9 loopback lo
[root@localhost ~]# hostnamectl hostname lvs
配置仅主机模式网卡（用于虚拟IP，便于用户访问）
[root@localhost ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.204.200/24 ipv4.gateway 192.168.204.2 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@localhost ~]# nmcli c up ens160
安装 ipvsamd
为了防止等会不能上外网，先把 ipvsamd 软件安装上
[root@lvs ~]# dnf install ipvsadm -y
配置NAT模式网卡（作为后端真实主机的网关）
[root@lvs ~]# nmcli c modify ens224 ipv4.method manual ipv4.addresses 192.168.88.8/24 ipv4.gateway 192.168.88.2 connection.autoconnect yes
[root@lvs ~]# nmcli c up ens224
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
[root@lvs ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
# 仅主机
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:1b brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.204.200/24 brd 192.168.204.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe26:e11b/64 scope link noprefixroute valid_lft forever preferred_lft forever
# NAT
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:25 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.88.8/24 brd 192.168.88.255 scope global noprefixroute ens224valid_lft forever preferred_lft foreverinet6 fe80::54c2:3ed3:5085:89a0/64 scope link noprefixroute valid_lft forever preferred_lft forever
2.2.3 配置客户端（仅主机）
客户端的网络采用的是仅主机模式。
[root@localhost ~]# hostnamectl hostname client
[root@localhost ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.204.100/24 ipv4.gateway 192.168.204.2 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@localhost ~]# nmcli c up ens160
[root@client ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:11:46:49
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.10.100/24
IP4.GATEWAY: 192.168.10.2
IP4.ROUTE[1]: dst = 192.168.204.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.204.2, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:fe11:4649/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
2.2.4 启动ipvsadm服务（lvs）
[root@lvs ~]# ipvsadm-save > /etc/sysconfig/ipvsadm
[root@lvs ~]# systemctl start ipvsadm
在客户端上访问 VIP 来测试是否能够成功访问到后端的 RS 服务器
#在客户端上访问 VIP 来测试是否能够成功访问到后端的 RS 服务器
[root@client ~]# curl 192.168.204.200
curl: (7) Failed to connect to 192.168.10.200 port 80: Connection refused
#在 lvs 服务器中可以访问：
[root@lvs ~]# curl 192.168.204.200
192.168.88.17
[root@lvs ~]# curl 192.168.204.200
192.168.88.7
# 由于没有做 LVS 规则匹配
2.2.5 LVS 规则匹配
由于没有做 LVS 规则匹配
# 配置一条规则
[root@lvs ~]# ipvsadm -A -t 192.168.10.200:80 -s rr
-A:虚拟的服务器地址
-t:指定虚拟的主机的地址
-s:算法 rr# 为规则增加RS真实服务器
[root@lvs ~]# ipvsadm -a -t 192.168.10.200:80 -r 192.168.72.7:80 -m -w 2
[root@lvs ~]# ipvsadm -a -t 192.168.10.200:80 -r 192.168.72.17:80 -m -w 2
-a:虚拟的服务器地址
-r:真实的服务器地址
-m:NAT模式-Masq（不指定即为DR模式-route）
-W:权重#去除规则：-d
#删除：-C[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.10.200:80 rr-> 192.168.72.7:80 Masq 2 0 0 -> 192.168.72.17:80 Masq 2 0 0 # 配置完后，重启服务器
[root@lvs ~]# systemctl restart ipvsadm
2.2.6 功能测试
客户端测试
[root@lvs ~]# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
不报错，但是没有数据返回。原因是需要配置内核转发参数
net.ipv4.ip_forward=1
NAT模式内核参数配置（LVS主机）
[root@lvs ~]# vim /etc/sysctl.conf
[root@lvs ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward=1 #添加内容# 执行如下的命令来生效
[root@lvs ~]# sysctl -p
net.ipv4.ip_forward = 1
[root@lvs ~]# systemctl restart ipvsadm
客户端再次测试
[root@client ~]# curl 192.168.204.200
192.168.88.17
[root@client ~]# curl 192.168.204.200
192.168.88.7
三、DR
3.1 DR工作模式
DR工作模式：
在LVS-DR模式下，负载均衡器仅修改请求报文的目标MAC地址，而不修改IP地址。负载均衡器将请求发送到后端服务器后，后端服务器直接将响应报文发回客户端。由于不修改IP地址，该模式具有较高的处理效率。
3.2 DR模式单网段案例
架构结构：
节点规划：
| 主机 | 角色 | 系统 | 网络 | IP |
|---|---|---|---|---|
| client | client | redhat 9.5 | 仅主机 | 192.168.98.100/24 |
| lvs | lvs | redhat 9.5 | 仅主机 NAT | 192.168.98.8/24 VIP:192.168.98.100/32 |
| nginx | rs1 | redhat 9.5 | NAT | 192.168.98.7/24 VIP:192.168.98.100/32 |
| nginx | rs2 | redhat 9.5 | NAT | 192.168.98.17/24 VIP:192.168.98.100/32 |
| router | 路由 | redhat 9.5 | 仅主机 NAT | 192.168.86.130/24、192.168.98.135/24 |
所有主机的防火墙和Selinux都要关闭
systemctl disable --now firewalld
临时关闭Selinux
setenforce 0
永久关闭Selinux
sed -i "s/SELINUX=enforcing/SELINUX=permissive"
3.2.1 配置路由
需要两块网卡，一块为仅主机模式，一块为NAT模式
[root@router ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:65:57:9e brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.86.130/24 brd 192.168.86.255 scope global dynamic noprefixroute ens160valid_lft 1521sec preferred_lft 1521secinet6 fe80::20c:29ff:fe65:579e/64 scope link noprefixroute valid_lft forever preferred_lft forever
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:65:57:a8 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.98.135/24 brd 192.168.98.255 scope global dynamic noprefixroute ens224valid_lft 1521sec preferred_lft 1521secinet6 fe80::4144:bf6f:b3ce:99b8/64 scope link noprefixroute valid_lft forever preferred_lft forever
- 第一块网卡（ens160）：
不需要上网，所以dns和gateway不需要配置
[root@router ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.86.200/24 connection.autoconnect yes
[root@router ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@router ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:65:57:9E
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/4
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.86.200/24
IP4.GATEWAY: --
IP4.ROUTE[1]: dst = 192.168.86.0/24, nh = 0.0.0.0, mt = 102
IP6.ADDRESS[1]: fe80::20c:29ff:fe65:579e/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
- 第二块网卡（ens224）：
#修改网卡连接名称
[root@router ~]# nmcli c show
NAME UUID TYPE DEVICE
Wired connection 1 bac67df3-eb21-31f8-bb92-92f73e2470e6 ethernet ens224
ens160 80630323-1c6a-381f-817b-4d1d206850e3 ethernet ens160
lo 4725cdc0-d053-4703-952b-a33bb54887b9 loopback lo
[root@router ~]# nmcli c modify 'Wired connection 1' connection.id ens224
[root@router ~]# nmcli c show
NAME UUID TYPE DEVICE
ens224 bac67df3-eb21-31f8-bb92-92f73e2470e6 ethernet ens224
ens160 80630323-1c6a-381f-817b-4d1d206850e3 ethernet ens160
lo 4725cdc0-d053-4703-952b-a33bb54887b9 loopback lo
#为了后续安装软件，配置网关和dns
[root@router ~]# nmcli c modify ens224 ipv4.method manual ipv4.addresses 192.168.98.200/24 ipv4.gateway 192.168.98.2 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@router ~]# nmcli c up ens224
[root@router ~]# nmcli d show ens224
GENERAL.DEVICE: ens224
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:65:57:A8
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens224
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/5
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.200/24
IP4.GATEWAY: 192.168.98.2
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 103
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.2, mt = 103
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::4144:bf6f:b3ce:99b8/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
3.2.2 配置RS真实服务器
- rs1
[root@rs1 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.7/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs1 ~]# nmcli c up ens160
[root@rs1 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:BA:BD:60
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.7/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:feba:bd60/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
配置IP后ping不了上网
[root@rs1 ~]# ping www.baidu.com
^C^C
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs1 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.3 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=32.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 28.316/30.484/32.653/2.168 ms
#安装好软件后改回网关
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
安装nginx
[root@rs1 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs1 ~]# dnf install nginx -y
[root@rs1 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs1 ~]# systemctl start nginx
[root@rs1 ~]# curl localhost
rs1 192.168.98.7
[root@rs1 ~]# curl 192.168.98.7
rs1 192.168.98.7
#路由也可访问
[root@router ~]# curl 192.168.98.7
rs1 192.168.98.7
- rs2:
[root@rs2 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.17/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs2 ~]# nmcli c up ens160
[root@rs2 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:38:37:87
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.17/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:fe38:3787/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
IP配置后不能上网
[root@rs2 ~]# ping wwww.baidu.com
^C
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs2 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.8 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=30.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 28.807/29.757/30.707/0.950 ms
#安装好软件nginx后，将网关改回来
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
- 安装nginx
[root@rs2 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs2 ~]# dnf install nginx -y
[root@rs2 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs2 ~]# systemctl start nginx
[root@rs2 ~]# curl localhost
rs2 192.168.98.17
[root@rs2 ~]# curl 192.168.98.17
rs2 192.168.98.17
#router服务器访问
[root@router ~]# curl 192.168.98.17
rs2 192.168.98.17
3.2.3 配置LVS
[root@client ~]# hostnamectl hostname lvs
[root@client ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.8/24 ipv4.gateway 192.168.98.200 connection.autoconnect yes
[root@client ~]# nmcli c up ens160
[root@lvs ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:26:E1:1B
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.8/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP6.ADDRESS[1]: fe80::20c:29ff:fe26:e11b/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
- 安装ipvsadm
[root@lvs ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@lvs ~]# dnf install ipvsadm -y
# 初始化文件
[root@lvs ~]# ipvsadm-save -n > /etc/sysconfig/ipvsadm
[root@lvs ~]# vim /usr/lib/systemd/system/ipvsadm.service
- 配置虚拟IP（VIP）
[root@lvs ~]# ip addr add 192.168.98.100/32 dev lo
[root@lvs ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3.2.4 给 RS 服务增加 VIP
- RS1：
[root@rs1 ~]# ip addr add 192.168.98.100/32 dev lo
[root@rs1 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
- RS2：
[root@rs2 ~]# ip addr add 192.168.98.100/32 dev lo
[root@rs2 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3.2.5 在 RS 上增加内核参数（防止IP冲突）
IP全部一样会产生冲突
[root@rs1 ~]# vim /etc/sysctl.conf
[root@rs1 ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
[root@rs1 ~]# sysctl -p #让配置生效
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
DR模式内核参数（RS主机）
[root@rs2 ~]# vim /etc/sysctl.conf
[root@rs2 ~]# cat /etc/sysctl.conf
....
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
[root@rs2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
3.2.6 配置 LVS 规则
[root@lvs ~]# systemctl start ipvsadm
# 配置规则
[root@lvs ~]# ipvsadm -A -t 192.168.98.100:80 -s wrr
wrr：带权重的轮询
-p 120（这是时间），强制持久化
[root@lvs ~]# ipvsadm -a -t 192.168.98.100:80 -r 192.168.98.7:80 -g -w 3
[root@lvs ~]# ipvsadm -a -t 192.168.98.100:80 -r 192.168.98.17:80 -g -w 1
-g：指定DR模式
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 wrr-> 192.168.98.7:80 Route 3 0 0 -> 192.168.98.17:80 Route 1 0 0
[root@lvs ~]# systemctl restart ipvsadm
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 wrr-> 192.168.98.7:80 Route 3 0 0 -> 192.168.98.17:80 Route 1 0 0
- 路由主机
[root@router ~]# vim /etc/sysctl.conf
[root@router ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward=1
[root@router ~]# sysctl -p
net.ipv4.ip_forward = 1
3.2.7 测试
- 客户端
#3:1 因为lvs配置时使用的权重wrr
[root@client ~]# while true; do curl 192.168.98.100;done;
rs1 192.168.98.7
rs1 192.168.98.7
rs2 192.168.98.17
rs1 192.168.98.7
rs1 192.168.98.7
rs1 192.168.98.7
rs2 192.168.98.17
- router
[root@router ~]# curl 192.168.98.100
rs2 192.168.98.17
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs2 192.168.98.17
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@router ~]# curl 192.168.98.100
rs2 192.168.98.17
3.3 DR多网段
3.3.1 配置路由
需要两块网卡，一块为仅主机模式，一块为NAT模式
[root@router ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:65:57:9e brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.86.130/24 brd 192.168.86.255 scope global dynamic noprefixroute ens160valid_lft 1521sec preferred_lft 1521secinet6 fe80::20c:29ff:fe65:579e/64 scope link noprefixroute valid_lft forever preferred_lft forever
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:65:57:a8 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.98.135/24 brd 192.168.98.255 scope global dynamic noprefixroute ens224valid_lft 1521sec preferred_lft 1521secinet6 fe80::4144:bf6f:b3ce:99b8/64 scope link noprefixroute valid_lft forever preferred_lft forever
- 第一块网卡（ens160）：
不需要上网，所以dns和gateway不需要配置
[root@router ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.86.200/24 connection.autoconnect yes
[root@router ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@router ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:65:57:9E
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/4
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.86.200/24
IP4.GATEWAY: --
IP4.ROUTE[1]: dst = 192.168.86.0/24, nh = 0.0.0.0, mt = 102
IP6.ADDRESS[1]: fe80::20c:29ff:fe65:579e/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
- 第二块网卡（ens224）：
#修改网卡连接名称
[root@router ~]# nmcli c show
NAME UUID TYPE DEVICE
Wired connection 1 bac67df3-eb21-31f8-bb92-92f73e2470e6 ethernet ens224
ens160 80630323-1c6a-381f-817b-4d1d206850e3 ethernet ens160
lo 4725cdc0-d053-4703-952b-a33bb54887b9 loopback lo
[root@router ~]# nmcli c modify 'Wired connection 1' connection.id ens224
[root@router ~]# nmcli c show
NAME UUID TYPE DEVICE
ens224 bac67df3-eb21-31f8-bb92-92f73e2470e6 ethernet ens224
ens160 80630323-1c6a-381f-817b-4d1d206850e3 ethernet ens160
lo 4725cdc0-d053-4703-952b-a33bb54887b9 loopback lo
#为了后续安装软件，配置网关和dns
[root@router ~]# nmcli c modify ens224 ipv4.method manual ipv4.addresses 192.168.98.200/24 ipv4.gateway 192.168.98.2 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@router ~]# nmcli c up ens224
[root@router ~]# nmcli d show ens224
GENERAL.DEVICE: ens224
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:65:57:A8
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens224
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/5
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.200/24
IP4.GATEWAY: 192.168.98.2
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 103
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.2, mt = 103
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::4144:bf6f:b3ce:99b8/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
3.3.2 配置RS真实服务器
- rs1
[root@rs1 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.7/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs1 ~]# nmcli c up ens160
[root@rs1 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:BA:BD:60
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.7/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:feba:bd60/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
配置IP后ping不了上网
[root@rs1 ~]# ping www.baidu.com
^C^C
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs1 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.3 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=32.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 28.316/30.484/32.653/2.168 ms
#安装好软件后改回网关
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
安装nginx
[root@rs1 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs1 ~]# dnf install nginx -y
[root@rs1 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs1 ~]# systemctl start nginx
[root@rs1 ~]# curl localhost
rs1 192.168.98.7
[root@rs1 ~]# curl 192.168.98.7
rs1 192.168.98.7
#路由也可访问
[root@router ~]# curl 192.168.98.7
rs1 192.168.98.7
- rs2:
[root@rs2 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.17/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs2 ~]# nmcli c up ens160
[root@rs2 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:38:37:87
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.17/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:fe38:3787/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
IP配置后不能上网
[root@rs2 ~]# ping wwww.baidu.com
^C
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs2 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.8 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=30.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 28.807/29.757/30.707/0.950 ms
#安装好软件nginx后，将网关改回来
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
- 安装nginx
[root@rs2 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs2 ~]# dnf install nginx -y
[root@rs2 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs2 ~]# systemctl start nginx
[root@rs2 ~]# curl localhost
rs2 192.168.98.17
[root@rs2 ~]# curl 192.168.98.17
rs2 192.168.98.17
#router服务器访问
[root@router ~]# curl 192.168.98.17
rs2 192.168.98.17
3.3.3 配置LVS
[root@client ~]# hostnamectl hostname lvs
[root@client ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.8/24 ipv4.gateway 192.168.98.200 connection.autoconnect yes
[root@client ~]# nmcli c up ens160
[root@lvs ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:26:E1:1B
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.8/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP6.ADDRESS[1]: fe80::20c:29ff:fe26:e11b/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
- 安装ipvsadm
[root@lvs ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@lvs ~]# dnf install ipvsadm -y
# 初始化文件
[root@lvs ~]# ipvsadm-save -n > /etc/sysconfig/ipvsadm
[root@lvs ~]# vim /usr/lib/systemd/system/ipvsadm.service
- 配置虚拟IP（VIP）
[root@lvs ~]# ip addr add 192.168.98.100/32 dev lo
[root@lvs ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
3.3.4 增加 VIP
- router：
[root@router ~]# nmcli connection modify ens224 +ipv4.addresses 172.16.0.200/24
[root@router ~]# nmcli connection up ens224
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@router ~]# ip ad show ens224
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:65:57:a8 brd ff:ff:ff:ff:ff:ffaltname enp19s0inet 192.168.98.200/24 brd 192.168.98.255 scope global noprefixroute ens224valid_lft forever preferred_lft foreverinet 172.16.0.200/24 brd 172.16.0.255 scope global noprefixroute ens224valid_lft forever preferred_lft foreverinet6 fe80::4144:bf6f:b3ce:99b8/64 scope link noprefixroute
- RS1：
[root@rs1 ~]# ip address add 192.168.98.100 dev lo
[root@rs1 ~]# ip addr del 192.168.98.100/32 dev lo
[root@rs1 ~]# ip address add 172.16.0.100 dev lo
[root@rs1 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 172.16.0.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
- RS2：
[root@rs2 ~]# ip address add 192.168.98.100 dev lo
[root@rs2 ~]# ip address add 172.16.0.100 dev lo^C
[root@rs2 ~]# ip addr del 192.168.98.100/32 dev lo
[root@rs2 ~]# ip address add 172.16.0.100 dev lo
[root@rs2 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 172.16.0.100/32 scope global lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
- lvs：
[root@lvs ~]# ip address add 192.168.98.100 dev lo
[root@lvs ~]# ip addr del 192.168.98.100/32 dev lo
[root@lvs ~]# ip address add 172.16.0.100 dev lo
3.3.5 配置 LVS 规则
[root@lvs ~]# ipvsadm -E -t 172.16.0.100:80 -s wrr
No such service
[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ipvsadm -A -t 172.16.0.100:80 -s wrr
[root@lvs ~]# ipvsadm -a -t 172.16.0.100:80 -r 192.168.98.7:80 -g -w 3
[root@lvs ~]# ipvsadm -a -t 172.16.0.100:80 -r 192.168.98.17:80 -g -w 3
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.0.100:80 wrr-> 192.168.98.7:80 Route 3 0 0 -> 192.168.98.17:80 Route 3 0 0
[root@lvs ~]# systemctl restart ipvsadm^C
[root@lvs ~]# systemctl stop ipvsadm
[root@lvs ~]# systemctl restart ipvsadm
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.0.100:80 wrr-> 192.168.98.7:80 Route 3 0 0 -> 192.168.98.17:80 Route 3 0 0
3.3.6 测试
四、Tun
4.1 Tun 工作模式
在LVS-TUN模式下，负载均衡器通过IP隧道将请求报文发送到后端服务器。后端服务器解封装后处理请求，并直接将响应报文发回客户端。该模式适用于 地理位置分散的集群
4.2 Tun 模式配置
4.2.1
4.2.2 配置RS真实服务器
- rs1
[root@rs1 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.7/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs1 ~]# nmcli c up ens160
[root@rs1 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:BA:BD:60
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.7/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:feba:bd60/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
配置IP后ping不了上网
[root@rs1 ~]# ping www.baidu.com
^C^C
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs1 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.3 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=32.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 28.316/30.484/32.653/2.168 ms
#安装好软件后改回网关
[root@rs1 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs1 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
安装nginx
[root@rs1 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs1 ~]# dnf install nginx -y
[root@rs1 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs1 ~]# systemctl start nginx
[root@rs1 ~]# curl localhost
rs1 192.168.98.7
[root@rs1 ~]# curl 192.168.98.7
rs1 192.168.98.7
#路由也可访问
[root@router ~]# curl 192.168.98.7
rs1 192.168.98.7
- rs2:
[root@rs2 ~]# nmcli c modify ens160 ipv4.method manual ipv4.addresses 192.168.98.17/24 ipv4.gateway 192.168.98.200 ipv4.dns 223.5.5.5 connection.autoconnect yes
[root@rs2 ~]# nmcli c up ens160
[root@rs2 ~]# nmcli d show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:38:37:87
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.17/24
IP4.GATEWAY: 192.168.98.200
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.200, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:fe38:3787/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
IP配置后不能上网
[root@rs2 ~]# ping wwww.baidu.com
^C
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.2
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
[root@rs2 ~]# ping www.baidu.com
PING www.a.shifen.com (183.2.172.17) 56(84) bytes of data.
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=1 ttl=128 time=28.8 ms
64 bytes from 183.2.172.17 (183.2.172.17): icmp_seq=2 ttl=128 time=30.7 ms
^C
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 28.807/29.757/30.707/0.950 ms
#安装好软件nginx后，将网关改回来
[root@rs2 ~]# nmcli c m ens160 ipv4.gateway 192.168.98.200
[root@rs2 ~]# nmcli c up ens160
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5)
- 安装nginx
[root@rs2 ~]# mount /dev/sr0 /mnt
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@rs2 ~]# dnf install nginx -y
[root@rs2 ~]# echo $(hostname) $(hostname -I) > /usr/share/nginx/html/index.html
[root@rs2 ~]# systemctl start nginx
[root@rs2 ~]# curl localhost
rs2 192.168.98.17
[root@rs2 ~]# curl 192.168.98.17
rs2 192.168.98.17
#router服务器访问
[root@router ~]# curl 192.168.98.17
rs2 192.168.98.17
4.2.3 配置 LVS
#查询内核里面的内容
[root@lvs ~]# lsmod | grep ipip
- 添加虚拟IP
[root@lvs ~]# ip addr del 192.168.98.100/32 dev lo
[root@lvs ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
[root@lvs ~]# ip addr add 192.168.98.100 dev tunl0
#当使用tunl0后，会自动开启ipip模式
[root@lvs ~]# lsmod | grep ipip
ipip 20480 0
tunnel4 16384 1 ipip
ip_tunnel 36864 1 ipip
[root@lvs ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
[root@lvs ~]# ip a show tunl0
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000link/ipip 0.0.0.0 brd 0.0.0.0inet 192.168.98.100/32 scope global tunl0valid_lft forever preferred_lft forever
- 多出了tul0模式
[root@lvs ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:26:e1:1b brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.98.8/24 brd 192.168.98.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe26:e11b/64 scope link noprefixroute valid_lft forever preferred_lft forever
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000link/ipip 0.0.0.0 brd 0.0.0.0inet 192.168.98.100/32 scope global tunl0valid_lft forever preferred_lft forever
4.2.4 给 RS 增加 VIP
- 删除虚拟IP命令
ip addr del 192.168.98.100/32 dev lo
- rs1:
[root@rs1 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
[root@rs1 ~]# ip addr add 192.168.98.100 dev tunl0
#最好用这种方式添加虚拟IP
[root@rs1 ~]# ifconfig tunl0 192.168.98.100 netmask 255.255.255.255 up
[root@rs1 ~]# lsmod | grep ipip
ipip 20480 0
tunnel4 16384 1 ipip
ip_tunnel 36864 1 ipip
[root@rs1 ~]# ip ad show tunl0
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000link/ipip 0.0.0.0 brd 0.0.0.0inet 192.168.98.100/32 scope global tunl0valid_lft forever preferred_lft forever
- rs2:
[root@rs2 ~]# ip ad show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
[root@rs2 ~]# ifconfig tunl0 192.168.98.100 netmask 255.255.255.255 up
[root@rs2 ~]# lsmod | grep ipip
ipip 20480 0
tunnel4 16384 1 ipip
ip_tunnel 36864 1 ipip
[root@rs2 ~]# ip ad show tunl0
3: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000link/ipip 0.0.0.0 brd 0.0.0.0inet 192.168.98.100/32 scope global tunl0valid_lft forever preferred_lft forever
4.2.5 在 RS 上增加内核参数
[root@rs1 ~]# vim /etc/sysctl.conf
[root@rs1 ~]# sysctl -p
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
Tun模式增加内核参数（RS主机）
[root@rs2 ~]# vim /etc/sysctl.conf
[root@rs2 ~]# sysctl -p
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.tunl0.rp_filter = 0
net.ipv4.conf.all.rp_filter = 0
4.2.6 lvs 配置规则
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@lvs ~]# ipvsadm -A -t 192.168.98.100:80 -s wrr
[root@lvs ~]# ipvsadm -a -t 192.168.98.100:80 -r 192.168.98.7:80 -i -w 3
[root@lvs ~]# ipvsadm -a -t 192.168.98.100:80 -r 192.168.98.17:80 -i -w 1
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 wrr-> 192.168.98.7:80 Tunnel 3 0 0 -> 192.168.98.17:80 Tunnel 1 0 0
[root@lvs ~]# systemctl restart ipvsadm
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 rr-> 192.168.98.7:80 Tunnel 3 0 0 -> 192.168.98.17:80 Tunnel 1 0 0
4.2.7 测试
#配置lvs没有选择权重w，所以1：1
[root@client ~]# curl 192.168.98.100
rs2 192.168.98.17
[root@client ~]# curl 192.168.98.100
rs1 192.168.98.7
[root@client ~]# curl 192.168.98.100
rs2 192.168.98.17
[root@client ~]# curl 192.168.98.100
rs1 192.168.98.7
4.2.8 报错
在增加虚拟IP的时候，最后使用这个命令，而不是ip address add这个命令
（这个命令可能导致后面客户端访问的时候报错）
[root@rs1 ~]# ifconfig tunl0 192.168.98.100 netmask 255.255.255.255 up[root@client ~]# curl 192.168.98.100
curl: (7) Failed to connect to 192.168.98.100 port 80: Connection refused
五、综合案例
只要是集群，就要时间同步
搭建 DR 模式
Keepalived是集群管理中保证集群高可用的一个服务软件，用来防止单点故障。
LVS工作在内核层，性能高效，能够处理大量并发请求，支持多种负载均衡算法和工作模式，适应不同的应用场景
Keepalive+LVS+Nginx+NFS高可用架构
- 实验目的：
- 客户端通过访问 LVS高可用集群VIP:192.168.98.100可以访问到NFS业务服务器的内容。
- 当VS-mastert负载均衡断开服务后，LVS-backup可以进行备用，不影响用户访问业务。
- 当WEB1或者WEB2关闭服务后，另外一台服务器可以正常访问业务。
每台主机都要关闭防火墙和SELinux
systemctl disable --now firewalld
临时关闭Selinux
setenforce 0
| 主机 | 角色 | 安装软件 | IP |
|---|---|---|---|
| nfs | NFS业务服务器 | nfs-utils | 192.168.98.138 |
| Web1 | Web服务 | nfs-utils、nginx | 192.168.98.41 |
| Web2 | Web服务 | nfs-utils、nginx | 192.168.98.42 |
| LVS-master | 负载均衡 | ipvsadm、keepalived | 192.168.98.31 VIP:192.168.98.100 |
| LVS-backup | 负载均衡 | ipvsadm、keepalived | 192.168.98.32 VIP:192.168.98.100 |
| client | 客户端 | 192.168.98. |
RS 的网关是 LVS 的 IP
1. NFS 业务服务器（192.168.98.138）
- 挂载，安装软件nfs-utils
- 创建共享目录
- 配置 /etc/exports，开启服务
systemctl start nfs-server - 暴露共享位置
showmount -t ip，本机IP（nfs主机IP）
要先启动服务再暴露共享位置，否则会报错RPC - 写一个.html文件共享给Web主机
echo $(hostname -I) > /nfs/web/index.html - 到nginx主机配置相关
# 1.挂载，安装软件nfs-utils
[root@nfs ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@nfs ~]# dnf install nfs-utils -y# 2.创建共享目录
[root@nfs ~]# mkdir /nfs/web -p# 3.配置 /etc/exports，开启服务
[root@nfs ~]# vim /etc/exports
[root@nfs ~]# cat /etc/exports
/nfs/web 192.168.98.41(rw,no_root_squash) 192.168.98.42(rw,no_root_squash)
或者
/nfs/web 192.168.98.*(rw,no_root_squash) #权限可写sync
[root@nfs ~]# systemctl start nfs-server# 4.暴露共享位置（要先启动服务在暴露共享位置，否则会报错RPC）
[root@nfs ~]# showmount -e 192.168.98.138
Export list for 192.168.98.138:
/nfs/web 192.168.98.42,192.168.98.41# 5. 写一个.html文件共享给Web主机
[root@nfs ~]# echo $(hostname -I) > /nfs/web/index.html
[root@nfs ~]# cd /nfs/web/
[root@nfs web]# ls
index.html
2. Web服务集群（搭建RS服务器）
- IP：
[root@Web1 ~]# nmcli device show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:BA:BD:60
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/2
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.41/24
IP4.GATEWAY: 192.168.98.2
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.2, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:feba:bd60/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
RS 的网关是 LVS 的 IP
- 挂载，安装软件 nfs-utils、nginx
- 挂载首页目录/usr/share/nginx/html/
mount -t nfs nfs主机IP:nfs共享目录 本机共享目录 - 启动服务，查看是否与nfs主机同步文件
- 增加内核参数
vim /etc/sysctl.conf
net.ipv4.ip_forward=1
sysctl -p
- Web1（192.168.98.41）
# 1.挂载，安装软件 nfs-utils、nginx
[root@Web1 ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@Web1 ~]# dnf install nginx nfs-utils -y# 2.挂载首页目录
[root@Web1 ~]# mount -t nfs 192.168.98.138:/nfs/web /usr/share/nginx/html/
[root@Web1 ~]# df /usr/share/nginx/html/
Filesystem 1K-blocks Used Available Use% Mounted on
192.168.98.138:/nfs/web 46587904 1754112 44833792 4% /usr/share/nginx/html# 3.启动服务，测试，查看是否与nfs主机同步文件
[root@Web1 ~]# systemctl start nfs-server nginx
[root@Web1 ~]# showmount -e 192.168.98.138
Export list for 192.168.98.138:
/nfs/web 192.168.98.42,192.168.98.41
[root@Web1 ~]# cd /usr/share/nginx/html/
[root@Web1 html]# ls
[root@Web1 html]# ls
index.html
[root@Web1 ~]# curl localhost
nfs 192.168.98.138
可以通过克隆进行高效率配置，关闭Web1服务器，对这台服务器进行克隆，（先启动Web2再重新启动Web1，防止IP冲突）操作改变主机名、修改IP
- Web2（192.168.98.42）
[root@Web2 ~]# mount /dev/sr0 /mnt/
mount: /mnt: WARNING: source write-protected, mounted read-only.
[root@Web2 ~]# dnf install nginx nfs-utils -y
[root@Web2 ~]# mount -t nfs 192.168.98.138:/nfs/web /usr/share/nginx/html/
[root@Web2 ~]# df /usr/share/nginx/html/
Filesystem 1K-blocks Used Available Use% Mounted on
192.168.98.138:/nfs/web 46587904 1754112 44833792 4% /usr/share/nginx/html
[root@Web2 ~]# systemctl start nfs-server
[root@Web2 ~]# systemctl start nginx
[root@Web2 ~]# ls /usr/share/nginx/html/
index.html
[root@Web2 ~]# curl localhost
nfs 192.168.98.138
- 查看nginx的网页文件存储目录
[root@Web1 ~]# rpm -ql nginx | grep html
/usr/share/nginx/html/404.html
/usr/share/nginx/html/50x.html
/usr/share/nginx/html/icons
/usr/share/nginx/html/icons/poweredby.png
/usr/share/nginx/html/index.html
/usr/share/nginx/html/nginx-logo.png
/usr/share/nginx/html/poweredby.png
/usr/share/nginx/html/system_noindex_logo.png
开机自启动
[root@Web1 ~]# systemctl enable nginx nfs-server
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-server.service → /usr/lib/systemd/system/nfs-server.service.
[root@Web2 ~]# systemctl enable nginx nfs-server
Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /usr/lib/systemd/system/nginx.service.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-server.service → /usr/lib/systemd/system/nfs-server.service.
# 重启主机后，查看开机自启动是否开启成功
[root@Web1 ~]# ps -ef | grep nginx
root 1742 1 0 13:28 ? 00:00:00 nginx: master process /usr/sbin/nginx
nginx 1743 1742 0 13:28 ? 00:00:00 nginx: worker process
nginx 1744 1742 0 13:28 ? 00:00:00 nginx: worker process
nginx 1745 1742 0 13:28 ? 00:00:00 nginx: worker process
nginx 1746 1742 0 13:28 ? 00:00:00 nginx: worker process
root 2142 1501 0 15:07 pts/0 00:00:00 grep --color=auto nginx
自动挂载
- 写自动挂载的目录
写自动挂载文件：
[root@Web1 ~]# vim /etc/sysctl.d/
[root@Web1 ~]# vim /etc/fstab
[root@Web1 ~]# vim /etc/rc.d/init.d/
[root@Web1 ~]# vim /etc/fstab
[root@Web1 ~]# cat /etc/fstab
........
/dev/mapper/rhel-root / xfs defaults 0 0
UUID=a656d423-6d9a-4a0a-b794-9161d8d66b0b /boot xfs defaults 0 0
UUID=EDBD-EDDF /boot/efi vfat umask=0077,shortname=winnt 0 2
/dev/mapper/rhel-swap none swap defaults 0 0
192.168.98.138:/nfs/web /usr/share/nginx/html/ nfs defaults 0 0
[root@Web1 ~]# systemctl daemon-reload
[root@Web1 ~]# mount -a #挂载全部
配置nginx（为了区分Web1与Web2访问的文件内容）
[root@nfs ~]# cd /nfs/web/
[root@nfs web]# ls
index.html
[root@nfs web]# mv index.html index1.html
[root@nfs web]# echo "Web1 index.html" > index1.html
[root@nfs web]# ls
index1.html
[root@nfs web]# echo "Web2 index.html" > index2.html
[root@nfs web]# ls
index1.html index2.html
现在共享目录下有两个文件，现在访问会报错
[root@Web1 ~]# curl localhost
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.20.1</center>
</body>
</html>
[root@Web1 ~]# vim /etc/nginx/conf.d/web1.conf
[root@Web1 ~]# cat /etc/nginx/conf.d/web1.conf
server {listen 80;server_name 192.168.98.41;location / {root /usr/share/nginx/html;index index1.html;}
}
[root@Web1 ~]# systemctl restart nginx
[root@Web1 ~]# curl 192.168.98.41
Web1 index.html
[root@Web1 ~]# curl localhost #localhost代表本机，我们配置nginx时用的时IP
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.20.1</center>
</body>
</html>
3. LVS主机（Keepalived+lvs）
- IP：
[root@master ~]# nmcli device show ens160
GENERAL.DEVICE: ens160
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 00:0C:29:2A:3F:65
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected)
GENERAL.CONNECTION: ens160
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/2
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 192.168.98.31/24
IP4.GATEWAY: 192.168.98.2
IP4.ROUTE[1]: dst = 192.168.98.0/24, nh = 0.0.0.0, mt = 100
IP4.ROUTE[2]: dst = 0.0.0.0/0, nh = 192.168.98.2, mt = 100
IP4.DNS[1]: 223.5.5.5
IP6.ADDRESS[1]: fe80::20c:29ff:fe2a:3f65/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 1024
RS 的网关是 LVS 的 IP
- 安装 ipvsadm、keepalived
- 初始化文件
ipvsadm-save -n > /etc/sysconfig/ipvsadm，启动服务
systemctl start ipvsadm - 添加虚拟 IP
ifconfig NAT 192.168.98.100 netmask 255.255.255.255 up
ip addr add 192.168.98.100 dev NAT
（在lvs主机curl虚拟IP） - 通过在keepalived配置文件中的内容，已经配置了ipvsadm的相关内容
- LVS-master（192.168.98.31）
# 1.安装 ipvsadm
[root@master ~]# dnf install keepalived ipvsadm -y# 2.初始化文件 ipvsadm-save -n > /etc/sysconfig/ipvsadm 启动服务
[root@master ~]# ipvsadm-save -n > /etc/sysconfig/ipvsadm
[root@master ~]# systemctl start ipvsadm# 3.配置keepalived
[root@master ~]# vim /etc/keepalived/keepalived.conf
[root@master ~]# cat /etc/keepalived/keepalived.conf
global_defs {router_id lvs-master
}vrrp_instance VI_1 {state MASTERinterface ens160virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.98.100}
}
#配置lvs，需要指定VIP地址
virtual_server 192.168.98.100 80 {delay_loop 6 #健康检查的间隔时间，单位为秒lb_algo rr #负载均衡的算法，rr表示轮询，wrr：带权重lb_kind DR #负载均衡的模式，此处为DR模式，支持的模式：NAT、DR、TUNpersistence_timeout 50 #持久化时间，默认为秒
# 此处的配置相当于ipvsadm -A -t 协议（t：tcp协议） 192.168.98.100:80 --s wrr -p 50 #好处：让会话一直保持，设置为0，则表示不持久化protocol TCP #负载协议#配置真实服务器，配置方式：IP 端口号 相当于 ipvsadm -a -t 192.168.98.100:80 -g -w 1,-g：DRreal_server 192.168.98.41 80 {weight 1 #权重，默认为1TCP_CHECK { #检测connect_timeout 3 #连接时间，单位为秒，即3秒中如果未连通，则表示此主机服务挂了retry 3 #重试次数delay_before_retry 3 #重试间隔时间}}real_server 192.168.98.42 80 {weight 1TCP_CHECK {connect_timeout 3retry 3delay_before_retry 3}}
}
# 这里面配置的东西，相当于配置了lvsadm
[root@master ~]# systemctl restart ipvsadm keepalived# 4.配置lvs
[root@master ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 rr persistent 50 #加了持久-> 192.168.98.41:80 Route 1 0 0 -> 192.168.98.42:80 Route 1 0 0- LVS-backup（192.168.98.32）
[root@backup ~]# dnf install keepalived ipvsadm -y
[root@backup ~]# ipvsadm-save -n > /etc/sysconfig/ipvsadm
[root@backup ~]# systemctl start ipvsadm
[root@backup ~]# systemctl start keepalived
[root@backup ~]# vim /etc/keepalived/keepalived.conf
[root@backup ~]# cat /etc/keepalived/keepalived.conf
global_defs {router_id lvs-backup
}vrrp_instance VI_1 {state BACKUPinterface ens160virtual_router_id 51priority 90advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.98.100}
}virtual_server 192.168.98.100 80 {delay_loop 6lb_algo rrlb_kind DRpersistence_timeout 50protocol TCPreal_server 192.168.98.41 80 {weight 1TCP_CHECK {connect_timeout 3retry 3delay_before_retry 3}}real_server 192.168.98.42 80 {weight 1TCP_CHECK {connect_timeout 3retry 3delay_before_retry 3}}
}
[root@backup ~]# systemctl restart keepalived ipvsadm
[root@backup ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 rr persistent 50-> 192.168.98.41:80 Route 1 0 0 -> 192.168.98.42:80 Route 1 0 0
4. 回到Web主机，修改
- 添加虚拟IP
ifconfig lo:1 192.168.98.100 netmask 255.255.255.255 broadcast 192.168.98.100 up - 配置内核参数 /etc/sysctl.conf
- 增加路由
route add -host 192.168.98.100 dev lo:1
route -n
[root@Web1 ~]# ifconfig lo:1 192.168.98.100 netmask 255.255.255.255 broadcast 192.168.98.100 up
[root@Web1 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 brd 192.168.98.100 scope global lo:1valid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
[root@Web2 ~]# ifconfig lo:1 192.168.98.100 netmask 255.255.255.255 broadcast 192.168.98.100 up
[root@Web2 ~]# ip a show lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet 192.168.98.100/32 brd 192.168.98.100 scope global lo:1valid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
- 内核参数
[root@Web1 ~]# cat >> /etc/sysctl.conf <<EOF
> net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.ip_forward=0
> EOF
[root@Web1 ~]# sysctl -p
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.ip_forward = 0
[root@Web2 ~]# cat >> /etc/sysctl.conf <<EOF
> net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.ip_forward=0
> EOF
[root@Web2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.ip_forward = 0
- 增加路由
[root@Web1 ~]# route add -host 192.168.98.100 dev lo:1
[root@Web1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.98.2 0.0.0.0 UG 100 0 0 ens160
192.168.98.0 0.0.0.0 255.255.255.0 U 100 0 0 ens160
192.168.98.100 0.0.0.0 255.255.255.255 UH 0 0 0 lo
[root@Web2 ~]# route add -host 192.168.98.100 dev lo:1
[root@Web2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.98.2 0.0.0.0 UG 100 0 0 ens160
192.168.98.0 0.0.0.0 255.255.255.0 U 100 0 0 ens160
192.168.98.100 0.0.0.0 255.255.255.255 UH 0 0 0 lo
5. 客户端测试(浏览器http://192.168.98.100)
停止nginx服务
- 停掉Web1：
[root@Web1 ~]# systemctl stop nginx# ipvsadm服务：
[root@master ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 rr persistent 50-> 192.168.98.42:80 Route 1 0 0
不可访问，访问报错
- 重启
[root@Web1 ~]# systemctl start nginx
[root@master ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.98.100:80 rr persistent 50-> 192.168.98.41:80 Route 1 0 0 -> 192.168.98.42:80 Route 1 0 0
停止keepalived服务（高可用）
当前状态所有主机，所有服务全部开启的
#虚拟IP在master主机上
[root@master ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:2a:3f:65 brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.98.31/24 brd 192.168.98.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe2a:3f65/64 scope link noprefixroute valid_lft forever preferred_lft forever
[root@backup ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:25:66:fb brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.98.32/24 brd 192.168.98.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe25:66fb/64 scope link noprefixroute valid_lft forever preferred_lft forever
- 停止master主机的keepalived
[root@master ~]# systemctl stop keepalived
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:2a:3f:65 brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.98.31/24 brd 192.168.98.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe2a:3f65/64 scope link noprefixroute valid_lft forever preferred_lft forever#虚拟IP漂移到backup主机
[root@backup ~]# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:0c:29:25:66:fb brd ff:ff:ff:ff:ff:ffaltname enp3s0inet 192.168.98.32/24 brd 192.168.98.255 scope global noprefixroute ens160valid_lft forever preferred_lft foreverinet 192.168.98.100/32 scope global ens160valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe25:66fb/64 scope link noprefixroute valid_lft forever preferred_lft forever
仍旧都可访问