Pre
双机热备是指两台机器都在运行,但并不是两台机器都同时在提供服务。
环境信息
节点 | 软件 | IP |
节点A | keepalived + tomcat | 192.168.126.137 |
节点B | keepalived + tomcat | 192.168.126.138 |
虚拟IP | - | 192.168.126.200 |
官网
https://keepalived.readthedocs.io/en/latest/#
原理
keepalived用于保证环境高可用,防止单点故障。
keepalived主要有三个模块,分别是core、check和VRRP。
- core核心模块,负责主进程的启动、维护以及全局配置文件的加载和解析
- check负责健康检查,包括常见的各种检查方式
- VRRP模块实现VRRP协议
VRRP(Virtual Router Redundancy Protocol) 即虚拟路由冗余协议,实现路由器高可用的协议,即将N台提供相同功能的路由器组成一个路由器组,这个组里面有一个master和多个backup,master上面有一个对外提供服务的vip(该路由器所在局域网内其他机器的默认路由为该vip),master会发组播,当backup收不到VRRP包时就认为master宕掉了,这时就需要根据VRRP的优先级来选举一个backup当master。这样的话就可以保证路由器的高可用了。
keepalived 安装
两个节点均需要安装, 在137 和 138 节点上安装 ,执行 yum install keepalived
137 安装 keepalived
[root@localhost ~]# yum install keepalived Loaded plugins: fastestmirror ....... ....... Complete! [root@localhost ~]#
138 安装 keepalived
[root@localhost ~]# yum install keepalived Loaded plugins: fastestmirror ....... ....... Complete! [root@localhost ~]#
tomcat 也都装下哈 ,用于验证,可以yum安装
CentOS 7 yum安装与配置 Tomcat 写的挺不错,没有的话可以参考
配置keepalived的配置文件/etc/keepalived/keepalived.conf
关键参数解读
state 主服务器为MASTER,其他服务器均为BACKUP interface 通信用的网卡,ip addr可以查看,以实际为准 virtual_router_id 主从服务器此ID必须一致 priority 优先级,主服务器必须大于其他服务器,数值越大优先级越高 authentication 主从服务器必须一致 virtual_ipaddress 虚拟IP地址,主从服务器必须一致,该VIP为客户端访问时使用的IP
137
! Configuration File for keepalived global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from example@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_tomcat { script "/etc/keepalived/tomcat_check.sh" interval 5 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state MASTER interface ens33 mcast_src_ip 192.168.126.137 virtual_router_id 51 priority 101 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.126.200 } track_script { chk_tomcat } }
138
! Configuration File for keepalived global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from example@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_tomcat { script "/etc/keepalived/tomcat_check.sh" interval 5 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state BACKUP interface ens33 mcast_src_ip 192.168.126.138 virtual_router_id 51 priority 99 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.126.200 } track_script { chk_tomcat } }
与主节点的配置上的区别,需要注意
state 表示一个是主机一个是备机,mcast_src_ip 当前服务器的ip,priority 主机要比备机的要高
检测脚本
!/bin/bash JAVA_PROCESS=`ps -C java --no-heading| wc -l` if [ $JAVA_PROCESS -eq 0 ];then echo "tomcat is stop" sleep 2 if [ `ps -C java --no-heading| wc -l` -eq 0 ];then systemctl stop keepalived.service fi fi
比较简单,根据实际情况调整
检测是否有tomcat在运行,如果没有的话,等2秒,如果还没有就把keepalived给停止掉,这样vip就能从主机master切换到备机BACKUP上了。
日志
可以查看 /var/log/messages 日志 观察切换和选主的过程。
状态发生变更,触发脚本
https://keepalived.readthedocs.io/en/latest/configuration_synopsis.html
! Configuration File for keepalived global_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from example@example.com smtp_server mail.example.com smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_tomcat { script "/etc/keepalived/tomcat_check.sh" interval 5 weight -5 fall 3 rise 2 } vrrp_instance VI_1 { state MASTER interface ens33 mcast_src_ip 192.168.126.137 virtual_router_id 51 priority 101 advert_int 2 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.126.200 } track_script { chk_tomcat } notify_master /etc/keepalived/a.sh notify_backup /etc/keepalived/b.sh
当节点从BACKUP 转为 MASTER的时候 ,执行 /etc/keepalived/a.sh
当节点从MASTER 转为 BACKUP的时候 ,执行 /etc/keepalived/b.sh
小结
MASTER和 BACKUP,预先分配好, 通过priority 来确定, 当MASTER宕机以后,BACKUP 没有收到 MASTER的 VRRP消息,BACKUP就认为 MASTER宕机了,这个时候就把自己选举为MASTER。
MASTER宕机,本节点的日志
Dec 30 01:40:01 localhost Keepalived[6784]: Stopping Dec 30 01:40:01 localhost systemd: Stopping LVS and VRRP High Availability Monitor... Dec 30 01:40:01 localhost Keepalived_vrrp[6786]: VRRP_Instance(VI_1) sent 0 priority Dec 30 01:40:01 localhost Keepalived_vrrp[6786]: VRRP_Instance(VI_1) removing protocol VIPs. Dec 30 01:40:01 localhost Keepalived_healthcheckers[6785]: Stopped Dec 30 01:40:02 localhost Keepalived_vrrp[6786]: Stopped Dec 30 01:40:02 localhost Keepalived[6784]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2 Dec 30 01:40:02 localhost systemd: Stopped LVS and VRRP High Availability Monitor.
BACKUP节点的日志
Dec 30 01:40:05 localhost Keepalived_vrrp[9173]: VRRP_Instance(VI_1) Entering MASTER STATE Dec 30 01:40:05 localhost Keepalived_vrrp[9173]: VRRP_Instance(VI_1) setting protocol VIPs.
当MASTER恢复以后,MASTER发送VRRP消息给BACKUP,这个时候虽然BACKUP是MASTER,但是 比较Prority后,发现自己的Prority小,会进入BACKUP模式,这个时候,之前设置的MASTER就又变成了MASTER了 。
MASTER恢复,本节点的日志
Dec 30 01:42:09 localhost Keepalived[6915]: Starting Healthcheck child process, pid=6916 Dec 30 01:42:09 localhost Keepalived[6915]: Starting VRRP child process, pid=6917 Dec 30 01:42:09 localhost Keepalived_healthcheckers[6916]: Opening file '/etc/keepalived/keepalived.conf'. Dec 30 01:42:09 localhost systemd: Started LVS and VRRP High Availability Monitor. Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: Registering Kernel netlink reflector Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: Registering Kernel netlink command channel Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: Registering gratuitous ARP shared channel Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: Opening file '/etc/keepalived/keepalived.conf'. Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: WARNING - default user 'keepalived_script' for script execution does not exist - please create. Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: SECURITY VIOLATION - scripts are being executed but script_security not enabled. Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: VRRP_Instance(VI_1) removing protocol VIPs. Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: Using LinkWatch kernel netlink reflector... Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)] Dec 30 01:42:09 localhost Keepalived_vrrp[6917]: VRRP_Script(chk_tomcat) succeeded Dec 30 01:42:10 localhost Keepalived_vrrp[6917]: VRRP_Instance(VI_1) Transition to MASTER STATE Dec 30 01:42:12 localhost Keepalived_vrrp[6917]: VRRP_Instance(VI_1) Entering MASTER STATE Dec 30 01:42:12 localhost Keepalived_vrrp[6917]: VRRP_Instance(VI_1) setting protocol VIPs. Dec 30 01:42:12 localhost Keepalived_vrrp[6917]: Sending gratuitous ARP on ens33 for 192.168.126.200
BACKUP节点的日志
Dec 30 01:42:09 localhost Keepalived_vrrp[9173]: /etc/keepalived/tomcat_check.sh exited with status 127 Dec 30 01:42:12 localhost Keepalived_vrrp[9173]: VRRP_Instance(VI_1) Received advert with higher priority 101, ours 94 Dec 30 01:42:12 localhost Keepalived_vrrp[9173]: VRRP_Instance(VI_1) Entering BACKUP STATE Dec 30 01:42:12 localhost Keepalived_vrrp[9173]: VRRP_Instance(VI_1) removing protocol VIPs.
Questions
exited due to signal 15
/var/log/messages 日志提示:/etc/keepalived/check_nginx.sh exited due to signal 15
调整如下参数
interval 5 #监控脚本的执行时间要大于advert_int advert_int 2 #心跳检查间隔时间
notify_master notify_backup未生效
检查 getenforce
getenforce 如果 开启的话,关闭 执行 setenforce 0