keepalived+Haproxy 高可用配置
目录
环境准备
一、配置准备
二、修改配置文件
khn-1 主机
修改keepalived配置文件
启用脚本编写
check_khn_nginx.sh
add_dnat.sh
remove_dnat.sh
haproxy配置文件
khn-2主机
修改keepalived配置文件
启用脚本编写
check_khn_nginx.sh
add_dnat.sh
remove_dnat.sh
haproxy配置文件
启用服务
khn-1
khn-2
查看
测试
故障模拟
通过keepalived双击热备、haproxy监控、nginx服务,实现nginx负载均衡
环境准备
keepalived双击热备 一主一从
两台nginx服务器 web1和web2
关闭安全上下文(四台主机)
setforce 0
关闭防火墙(四台主机)
systemctl stop firewalld
一、配置准备
keepalived + haproxy 端 (主和备)
KHN Triad
主命名为 khn-1
hostnamectl set-hostname khn-1
备命名为 khn-2
hostnamectl set-hostname khn-2
下载keepalived (主和备都下载)
yum -y install keepalived
下载haproxy (主和备都下载)
yum -y install haproxy
nginx端服务器
命名为 web1 和 web2
web1
yum -y install nginx
systemctl start nginx
echo web1 > /usr/share/nginx/html/index.html
web2
yum -y install nginx
systemctl start nginx
echo web2 > /usr/share/nginx/html/index.html
二、修改配置文件
keepalived配置文件
keepalived + haproxy 端
khn-1 主机
修改keepalived配置文件
[root@khn-1 ]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {router_id khn-1
}
vrrp_script check_khn_nginx { script /etc/keepalived/check_khn_nginx.sh #检测脚本interval 2 #执行间隔时间}vrrp_instance VI_1 {state MASTERinterface ens33virtual_router_id 51priority 100advert_int 3authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.100.109}track_script { #在实例中引用脚本 check_khn_nginx }
}#当本机成为 MASTER 时,添加 DNAT 规则
notify_master "/etc/keepalived/add_dnat.sh"
#当本机成为 BACKUP 时,删除 DNAT 规则
notify_backup "/etc/keepalived/remove_dnat.sh"
#当 keepalived 停止时,清理规则
notify_stop "/etc/keepalived/remove_dnat.sh"
启用脚本编写
check_khn_nginx.sh
[root@khn-1 ]# cd /etc/keepalived/
[root@khn-1 keepalived]# ls
add_dnat.sh check_khn_nginx.sh keepalived.conf keepalived.conf.sample remove_dnat.sh
[root@khn-1 keepalived]# vim check_khn_nginx.sh
#!/bin/bash#number=$(ps -C haproxy --no-headers | wc l)
#if [ $number -eq 0 ];then
# systemctl stop keepalived
# exit 1
#else
# exit 0
#fi
#Count1=`netstat -antp |grep -v grep |grep haproxy |wc -l` #统计进程是0或1(0没有1有)if [ $Count1 -eq 0 ]; thensystemctl restart haproxysleep 10Count2=`netstat -antp |grep -v grep |grep haproxy |wc -l`if [ $Count2 -eq 0 ]; then systemctl stop keepalived.service elseexit 0fi else exit 0fi
chmod +x check_khn_nginx.sh
add_dnat.sh
[root@khn-1 keepalived]# vim add_dnat.sh
#!/bin/bash
#
#将访问192.168.100.109:80 的流量 DNAT 到 192.168.100.100:80(khn-1)
ipvsadm -t nat -A PREROUTING -d 192.168.100.109 -p tcp --dprot 80 -j DNAT --to-destination 192.168.100.100:80
#转发监控页面 (1080) 流量到 192.168.100.100
ipvsadm -t nat -A PREROUTING -d 192.168.100.109 -p tcp --dprot 1080 -j DNAT --to-destination 192.168.100.100:1080
chmod +x add_dnat.sh
remove_dnat.sh
[root@khn-1 keepalived]# vim remove_dnat.sh
#!/bin/bash
# 清理 DNAT规则
iptables -t nat -F
chmod +x remove_dnat.sh
haproxy配置文件
[root@khn-1 ~]# cd /etc/haproxy/
[root@khn-1 haproxy]# ls
conf.d haproxy.bck1 haproxy.cfg haproxy.cfg.bck
[root@khn-1 haproxy]# vim haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------global #全局配置log 127.0.0.1 local2chroot /var/lib/haproxypidfile /var/run/haproxy.piduser haproxygroup haproxydaemonmaxconn 4000defaults #默认配置mode httplog globaloption httplogoption dontlognullretries 3timeout http-request 5stimeout queue 1mtimeout connect 5stimeout client 1mtimeout server 1mtimeout http-keep-alive 5stimeout check 5smaxconn 3000frontend main #虚拟ip:端口号 (前端配置)bind 192.168.100.100:80default_backend http_backbackend http_back #真实ip:端口号 (后端配置)balance roundrobinserver node1 192.168.100.10:80 checkserver node2 192.168.100.11:80 check
listen stats #定义监控页面 mode httpoption httplogbind 192.168.100.100:1080 #绑定端口1080 stats refresh 30s #每30秒更新监控数据 stats uri /stats #访问监控页面的uri stats realm HAProxy\ Stats #监控页面的认证提示 stats auth admin:admin #监控页面的用户名和密码
khn-2主机
修改keepalived配置文件
[root@khn-2 ]# vim /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs {router_id khn-2
}
vrrp_script check_khn_nginx { script /etc/keepalived/check_khn_nginx.sh #检测脚本interval 2 #执行间隔时间}vrrp_instance VI_1 {state MASTERinterface ens33virtual_router_id 51priority 80advert_int 3authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.100.109}track_script { #在实例中引用脚本 check_khn_nginx }
}#当本机成为 MASTER 时,添加 DNAT 规则
notify_master "/etc/keepalived/add_dnat.sh"
#当本机成为 BACKUP 时,删除 DNAT 规则
notify_backup "/etc/keepalived/remove_dnat.sh"
#当 keepalived 停止时,清理规则
notify_stop "/etc/keepalived/remove_dnat.sh"
启用脚本编写
check_khn_nginx.sh
[root@khn-2 ]# cd /etc/keepalived/
[root@khn-2 keepalived]# ls
add_dnat.sh check_khn_nginx.sh keepalived.conf keepalived.conf.sample remove_dnat.sh
[root@khn-2 keepalived]# vim check_khn_nginx.sh
#!/bin/bash#number=$(ps -C haproxy --no-headers | wc l)
#if [ $number -eq 0 ];then
# systemctl stop keepalived
# exit 1
#else
# exit 0
#fi
#Count1=`netstat -antp |grep -v grep |grep haproxy |wc -l` #统计进程是0或1(0没有1有)if [ $Count1 -eq 0 ]; thensystemctl restart haproxysleep 10Count2=`netstat -antp |grep -v grep |grep haproxy |wc -l`if [ $Count2 -eq 0 ]; then systemctl stop keepalived.service elseexit 0fi else exit 0fi
chmod +x check_khn_nginx.sh
add_dnat.sh
[root@khn-2 keepalived]# vim add_dnat.sh
#!/bin/bash
#
#将访问192.168.100.109:80 的流量 DNAT 到 192.168.100.101:80(khn-1)
ipvsadm -t nat -A PREROUTING -d 192.168.100.109 -p tcp --dprot 80 -j DNAT --to-destination 192.168.100.101:80
#转发监控页面 (1080) 流量到 192.168.100.101
ipvsadm -t nat -A PREROUTING -d 192.168.100.109 -p tcp --dprot 1080 -j DNAT --to-destination 192.168.100.101:1080
chmod +x add_dnat.sh
remove_dnat.sh
[root@khn-2 keepalived]# vim remove_dnat.sh
#!/bin/bash
# 清理 DNAT规则
iptables -t nat -F
chmod +x remove_dnat.sh
haproxy配置文件
[root@khn-2 ~]# cd /etc/haproxy/
[root@khn-2 haproxy]# ls
conf.d haproxy.bck1 haproxy.cfg haproxy.cfg.bck
[root@khn-2 haproxy]# vim haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------global #全局配置log 127.0.0.1 local2chroot /var/lib/haproxypidfile /var/run/haproxy.piduser haproxygroup haproxydaemonmaxconn 4000defaults #默认配置mode httplog globaloption httplogoption dontlognullretries 3timeout http-request 5stimeout queue 1mtimeout connect 5stimeout client 1mtimeout server 1mtimeout http-keep-alive 5stimeout check 5smaxconn 3000frontend main #虚拟ip:端口号 (前端配置)bind 192.168.100.101:80default_backend http_backbackend http_back #真实ip:端口号 (后端配置)balance roundrobinserver node1 192.168.100.10:80 checkserver node2 192.168.100.11:80 check
listen stats #定义监控页面 mode httpoption httplogbind 192.168.100.101:1080 #绑定端口1080 stats refresh 30s #每30秒更新监控数据 stats uri /stats #访问监控页面的uri stats realm HAProxy\ Stats #监控页面的认证提示 stats auth admin:admin #监控页面的用户名和密码
启用服务
khn-1
systemctl start haproxy
systemctl start keepalived
khn-2
systemctl start haproxy
systemctl start keepalived
查看
khn-1
[root@khn-1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:6a:66:62 brd ff:ff:ff:ff:ff:ffinet 192.168.100.100/24 brd 192.168.100.255 scope global noprefixroute ens33valid_lft forever preferred_lft foreverinet 192.168.100.109/32 scope global ens33valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe6a:6662/64 scope link noprefixroute valid_lft forever preferred_lft forever
khn-2
[root@khn-2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:75:83:bc brd ff:ff:ff:ff:ff:ffinet 192.168.100.101/24 brd 192.168.100.255 scope global noprefixroute ens33valid_lft forever preferred_lft foreverinet6 fe80::20c:29ff:fe75:83bc/64 scope link noprefixroute valid_lft forever preferred_lft forever
测试
网页访问192.168.100.109
会显示出 web1 和 web2 ,会来回切换
网页访问 192.168.100.109:1080/stats
故障模拟
停用(主)khn-1 的haproxy服务
systemctl stop haproxy
khn-2
[root@khn-2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000link/ether 00:0c:29:75:83:bc brd ff:ff:ff:ff:ff:ffinet 192.168.100.101/24 brd 192.168.100.255 scope global noprefixroute ens33valid_lft forever preferred_lft foreverinet 192.168.100.109/32 scope global ens33valid_lft forever preferred_lft forever
此时启用 (备),也能正常访问