NGINX高可用之keepalived+nginx主从模式+主主模式配置实践

本文涉及的产品
日志服务 SLS,月写入数据量 50GB 1个月
应用型负载均衡 ALB,每月750个小时 15LCU
传统型负载均衡 CLB,每月750个小时 15LCU
简介: NGINX高可用之keepalived+nginx主从模式+主主模式配置实践

七层负载均衡


七层就是基于URL等应用层信息的负载均衡。从第七层"应用层"开始,根据虚拟的url或IP,主机名接收请求,再转向相应的处理服务器。七层负载均衡器也称作七层交换机,即L7 switch(七层交换),OSI的最高层,应用层。此时,该Load Balancer能理解应用协议。如:HAProxy、Nginx等。


四层负载均衡


四层就是基于IP+端口的负载均衡,是在三次负载均衡的基础上,即从第四层"传输层"开始,使用"ip+port"接收请求,再转发到对应的机器。四层负载均衡器也称作四层交换机,即L4 switch(四层交换),在OSI第4层工作,此种Load Balance不理解应用协议(如HTTP/FTP/MySQL等等)。如:LVS、F5、深信服AD等。


nginx进程基于Master+Slave(worker)多进程模型,自身具有非常稳定的子进程管理功能。在Master进程分配模式下,Master进程永远不进行业务处理,只是进行任务分发,从而达到Master进程的存活高可靠性,Slave(worker)进程所有的业务信号都 由主进程发出,Slave(worker)进程所有的超时任务都会被Master中止,属于非阻塞式任务模型。


Keepalived是Linux下面实现VRRP备份路由的高可靠性运行件。基于Keepalived设计的服务模式能够真正做到主服务器和备份服务器故障时IP瞬间无缝交接。二者结合,可以构架出比较稳定的软件LB(LoadBalance)方案。


Keepalived是一个基于VRRP协议来实现的服务高可用方案,可以利用其来避免IP单点故障,类似的工具还有heartbeat、corosync、pacemaker。但是它一般不会单独出现,而是与其它负载均衡技术(如lvs、haproxy、nginx)一起工作来达到集群的高可用。


VRRP协议


VRRP全称 Virtual Router Redundancy Protocol,即 虚拟路由冗余协议。可以认为它是实现路由器高可用的容错协议,即将N台提供相同功能的路由器组成一个路由器组(Router Group),这个组里面有一个master和多个backup,但在外界看来就像一台一样,构成虚拟路由器,拥有一个虚拟IP(vip,也就是路由器所在局域网内其他机器的默认路由),占有这个IP的master实际负责ARP相应和转发IP数据包,组中的其它路由器作为备份的角色处于待命状态。master会发组播消息,当backup在超时时间内收不到vrrp包时就认为master宕掉了,这时就需要根据VRRP的优先级来选举一个backup当master,保证路由器的高可用。


在VRRP协议实现里,虚拟路由器使用 00-00-5E-00-01-XX 作为虚拟MAC地址,XX就是唯一的 VRID (Virtual Router IDentifier),这个地址同一时间只有一个物理路由器占用。在虚拟路由器里面的物理路由器组里面通过多播IP地址 224.0.0.18 来定时发送通告消息。每个Router都有一个 1-255 之间的优先级别,级别最高的(highest priority)将成为主控(master)路由器。通过降低master的优先权可以让处于backup状态的路由器抢占(pro-empt)主路由器的状态,两个backup优先级相同的IP地址较大者为master,接管虚拟IP。

【1】基础环境准备

两台服务器:192.168.88.129(主) 192.168.88.130(从),每台服务器上分别安装nginx 、keepalived。

首先要关闭防火墙、关闭selinux。


当然,你可以选择不关闭,但是可能会遇到各种奇怪问题。


安装keepalived可使用yum直接安装:yum -y install keepalived

默认keepalived.conf配置文件

! Configuration File for keepalived
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.200.16
        192.168.200.17
        192.168.200.18
    }
}
virtual_server 192.168.200.100 443 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP
    real_server 192.168.201.100 443 {
        weight 1
        SSL_GET {
            url {
              path /
              digest ff20ad2481f97b1754ef3e12ecd3a9cc
            }
            url {
              path /mrtg/
              digest 9b3a0c85a887a256d6939da88aabd8cd
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
virtual_server 10.10.10.2 1358 {
    delay_loop 6
    lb_algo rr 
    lb_kind NAT
    persistence_timeout 50
    protocol TCP
    sorry_server 192.168.200.200 1358
    real_server 192.168.200.2 1358 {
        weight 1
        HTTP_GET {
            url { 
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.200.3 1358 {
        weight 1
        HTTP_GET {
            url { 
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334c
            }
            url { 
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334c
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
virtual_server 10.10.10.3 1358 {
    delay_loop 3
    lb_algo rr 
    lb_kind NAT
    persistence_timeout 50
    protocol TCP
    real_server 192.168.200.4 1358 {
        weight 1
        HTTP_GET {
            url { 
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.200.5 1358 {
        weight 1
        HTTP_GET {
            url { 
              path /testurl/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl2/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            url { 
              path /testurl3/test.jsp
              digest 640205b7b0fc66c1ea91c463fac6334d
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

接下来就是要对该配置文件进行修改,该配置文件在/etc/keepalived/路径下。在默认的keepalive.conf里面还有 virtual_server,real_server 这样的配置,它是为lvs准备的。


首先开始配置主从模式,实例图如下:

【2】修改keepalived.conf配置文件

① 修改主机keepalived.conf配置

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.88.129 #这里修改为本机IP
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
#添加检测脚本
vrrp_script chk_http_port {
        script "/usr/local/nginx/nginx_check.sh"
        interval 2
        weight 2
}
vrrp_instance VI_1 {
    state MASTER   #主机这里是MASTER 从机是BACKUP
    interface ens33  #网卡
    virtual_router_id 51  # 主、从机的virtual_router_id必须相同
    priority 100   # 主备机取不同的优先级,主机优先级大
    advert_int 1  #心跳检测间隔时间
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.50   # VRRP 虚拟IP ;可换行输入多个进行绑定
    }
}



在默认的keepalive.conf里面还有 virtual_server,real_server 这样的配置,我们这用不到,它是为lvs准备的。

/usr/local/nginx/nginx_check.sh 脚本内容如下:

#!/bin/bash
counter=$(ps -C nginx --no-heading|wc -l)
if [ "${counter}" = "0" ]; then
    /usr/local/nginx/sbin/nginx
    sleep 2
    counter=$(ps -C nginx --no-heading|wc -l)
    if [ "${counter}" = "0" ]; then
        systemctl stop keepalived
    fi
fi


如果发现nginx进程不存在,则尝试启动;sleep2秒后再次检测,如果还是不存在则认为启动失败,就停止keepalived服务。


② 修改从机keepalived.conf配置文件

! Configuration File for keepalived
global_defs {
   notification_email {  #指定keepalived在发生事件时(比如切换)发送通知邮件的邮箱
     acassen@firewall.loc  #设置报警邮件地址,可以设置多个,每行一个。 需开启本机的sendmail服务
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc   #keepalived在发生诸如切换操作时需要发送email通知地址
   smtp_server 192.168.88.129   #指定发送email的smtp服务器
   smtp_connect_timeout 30       #设置连接smtp server的超时时间
   router_id LVS_DEVEL            #运行keepalived的机器的一个标识,通常可设为hostname。故障发生时,发邮件时显示在邮件主题中的信息。
}
vrrp_script chk_http_port {
        script "/usr/local/nginx/nginx_check.sh"
        interval 2   #检测脚本执行间隔时间
        weight 2    #设置当前服务器权重增量
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.50
    }
}


同样需要在/usr/local/nginx/路径下放上nginx_check.sh脚本文件。


③ 启动nginx、keepalived

/usr/local/nginx/sbin/nginx   #启动nginx
systemctl start keepalived  #启动keepalived


【3】测试

① 浏览器访问http://192.168.88.50/

192.168.88.129主nginx访问日志打印如下:


查看主服务器keepalived日志

Keepalived默认所有的日志都是写入到/var/log/message ,你可以使用命令 tail -f /var/log/messages|grep Keepalived 进行查看


尝试访问主服务器部署的页面http://192.168.88.50/edu/index.html:

主机192.168.88.129使用ip address命令检测可以发现虚拟ip已经飘到了主机上面:

② 停掉主机的keepalived

查看主机日志信息:

Jun 30 15:42:34 bogon Keepalived[5789]: Stopping
Jun 30 15:42:34 bogon Keepalived_healthcheckers[5791]: Stopped
Jun 30 15:42:34 bogon Keepalived_vrrp[5792]: VRRP_Instance(VI_1) sent 0 priority
Jun 30 15:42:34 bogon Keepalived_vrrp[5792]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 15:42:35 bogon Keepalived_vrrp[5792]: Stopped
Jun 30 15:42:35 bogon Keepalived[5789]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2


查看从机日志信息:

#当主服务器停掉后,从机将会切换为MASTER状态
Jun 30 13:26:14 bogon avahi-daemon[731]: Withdrawing address record for 192.168.88.50 on ens33.
Jun 30 13:28:55 bogon Keepalived_vrrp[2966]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:28:56 bogon avahi-daemon[731]: Registering new address record for 192.168.88.50 on ens33.IPv4.
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.50
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 13:29:01 bogon Keepalived_vrrp[2966]: Sending gratuitous ARP on ens33 for 192.168.88.50
#当主服

检测从机ip地址:

可以发现从机的keepalived已经切换为了MASTER状态,且从机的ip address检测时发现VIP绑定到了ens33上面。此时访问http://192.168.88.50/就会跑到从服务器的nginx上面!


再次启动主服务器的keepalived,查看主机日志:

主服务器的keepalived进入MASTER STATE,从服务器的keepalived进入BACKUP STATE:

#从服务器日志
Jun 30 15:01:24 bogon Keepalived_vrrp[1900]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 90
Jun 30 15:01:24 bogon Keepalived_vrrp[1900]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun 30 15:01:24 bogon Keepalived_vrrp[1900]: VRRP_Instance(VI_1) removing protocol VIPs.

③ 停掉主服务器的nginx,不停keepalived

则会执行脚本nginx_check.sh进行nginx启动!

总结

master没挂,则master占有vip且nginx运行在master上

master挂了,则slave抢占vip且在slave上运行nginx服务

如果master上的nginx服务挂了,则nginx会自动重启,重启失败后会自动关闭keepalived,这样vip资源也会转移到slave上。

master和slave两边都开启nginx服务,无论master还是slave,当其中的一个keepalived服务停止后,vip都会漂移到keepalived服务还在的节点上;

如果要想使nginx服务挂了,vip也漂移到另一个节点,则必须用脚本或者在配置文件里面用shell命令来控制。(nginx服务宕停后会自动启动,启动失败后会强制关闭keepalived,从而致使vip资源漂移到另一台机器上)


一种常见的主从热备应用实例如下图所示:

主从的缺点在于如果主机一直稳定,那么从机就会一直处于空闲状态,造成了资源的浪费。


【4】问题总结

① 不能完全停掉keepalived进程

使用yum 安装的keepalived,当使用命令systemctl stop keepalived 停掉keepalived服务时,使用ps命令检测发现还存,如下图示:


当使用命令systemctl status keepalived检测状态,会发现有一条警告信息,如下所示:

Jun 30 11:52:20 bogon systemd[1]: Stopped LVS and VRRP High Availability Monitor.
Jun 30 11:52:20 bogon systemd[1]: Stopping LVS and VRRP High Availability Monitor...
Jun 30 11:52:20 bogon systemd[1]: Starting LVS and VRRP High Availability Monitor...
Jun 30 11:52:20 bogon systemd[1]: Can't open PID file /var/run/keepalived.pid (yet?) after start: No such file or directory
Jun 30 11:52:20 bogon systemd[1]: Started LVS and VRRP High Availability Monitor.
Jun 30 12:30:57 bogon systemd[1]: Stopped LVS and VRRP High Availability Monitor.

查看其服务脚本信息vim /usr/lib/systemd/system/keepalived.service

[Unit]
Description=LVS and VRRP High Availability Monitor
After=syslog.target network-online.target
[Service]
Type=forking
PIDFile=/var/run/keepalived.pid
KillMode=process
EnvironmentFile=-/etc/sysconfig/keepalived
ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
[Install]
WantedBy=multi-user.target


KillMode=process的大致意思是当停止keepalived的时候只会停掉主进程,而主进程产生的子进程是不会被干掉的。而killmode的默认值是control-group,意思时所有进程都会被干掉,这里选择把这项注释掉。


重载配置

systemctl daemon-reload


杀掉keepalived所有进程,然后再次启动keepalived即可:

pkill -9 keepalived


② Unsafe permissions found for script ‘/usr/local/nginx/nginx_check.sh’.

说明你赋予的权限太高了,可以使用如下命令尝试:

chmod 755 /usr/local/nginx/nginx_check.sh


【5】keepalived与heartbeat/corosync等比较

Heartbeat、Corosync、Keepalived这三个集群组件我们到底选哪个好呢?

首先要说明的是,Heartbeat、Corosync是属于同一类型,Keepalived与Heartbeat、Corosync,根本不是同一类型的。


Keepalived使用的vrrp协议方式,虚拟路由冗余协议 (Virtual Router Redundancy Protocol,简称VRRP);Heartbeat或Corosync是基于主机或网络服务的高可用方式。


简单的说就是,Keepalived的目的是模拟路由器的高可用,Heartbeat或Corosync的目的是实现Service的高可用。

所以一般Keepalived是实现前端高可用,常用的前端高可用的组合有LVS+Keepalived、Nginx+Keepalived、HAproxy+Keepalived。


而Heartbeat或Corosync是实现服务的高可用。常见的组合有Heartbeat v3(Corosync)+Pacemaker+NFS+Httpd 实现Web服务器的高可用、Heartbeat v3(Corosync)+Pacemaker+NFS+MySQL 实现MySQL服务器的高可用。


总结一下,Keepalived中实现轻量级的高可用,一般用于前端高可用,且不需要共享存储,一般常用于两个节点的高可用。而Heartbeat(或Corosync)一般用于服务的高可用,且需要共享存储,一般用于多节点的高可用。这个问题我们说明白了。


那heartbaet与corosync又应该选择哪个好?


一般用corosync,因为corosync的运行机制更优于heartbeat,就连从heartbeat分离出来的pacemaker都说在以后的开发当中更倾向于corosync,所以现在corosync+pacemaker是最佳组合。


【6】主主模式配置实践

主主模式相对于主从模式而言区别在于,每个机器都互为主从。示意图如下:

① 修改192.168.88.129的keepalived.conf配置


添加如下配置:

vrrp_instance VI_2 {
    state BACKUP  #这里修改为BACKUP
    interface ens33
    virtual_router_id 52 #这里修改为52
    priority 90    #修改优先级
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.60  #绑定另外一个VIP
    }
track_script {
   chk_http_port
}
}

完整配置如下:

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {
        script "/usr/local/nginx/nginx_check.sh"
        interval 2
        weight 2
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
                192.168.88.50
    }
track_script {
   chk_http_port
}
}
vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 52
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.60
    }
track_script {
   chk_http_port
}
}

② 修改192.168.88.130的keepalived.conf配置

添加配置如下:

vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.60
    }
track_script {
   chk_http_port
}
}

完整配置如下:

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_http_port {
        script "/usr/local/nginx/nginx_check.sh"
        interval 2
        weight 2
}
vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
                192.168.88.50
    }
track_script {
   chk_http_port
}
}
vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.88.60
    }
track_script {
   chk_http_port
}
}

分别重启两台服务器上的keepalived服务,查看对应的日志信息。

192.168.88.129上keepalived日志信息如下:

Jun 30 17:00:13 bogon Keepalived[7503]: Stopping
Jun 30 17:00:13 bogon Keepalived_healthcheckers[7504]: Stopped
Jun 30 17:00:13 bogon Keepalived_vrrp[7505]: VRRP_Instance(VI_1) sent 0 priority
Jun 30 17:00:13 bogon Keepalived_vrrp[7505]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 17:00:14 bogon Keepalived_vrrp[7505]: Stopped
Jun 30 17:00:14 bogon Keepalived[7503]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Jun 30 17:00:14 bogon Keepalived[16827]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Jun 30 17:00:14 bogon Keepalived[16827]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:14 bogon Keepalived[16828]: Starting Healthcheck child process, pid=16830
Jun 30 17:00:14 bogon Keepalived[16828]: Starting VRRP child process, pid=16831
Jun 30 17:00:14 bogon Keepalived_healthcheckers[16830]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:14 bogon Keepalived_vrrp[16831]: Registering Kernel netlink reflector
Jun 30 17:00:14 bogon Keepalived_vrrp[16831]: Registering Kernel netlink command channel
Jun 30 17:00:14 bogon Keepalived_vrrp[16831]: Registering gratuitous ARP shared channel
Jun 30 17:00:14 bogon Keepalived_vrrp[16831]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:14 bogon Keepalived_vrrp[16831]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) removing protocol VIPs.
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: Using LinkWatch kernel netlink reflector...
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Entering BACKUP STATE
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jun 30 17:00:15 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.50
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:16 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:17 bogon Keepalived_vrrp[16831]: /usr/local/nginx/nginx_check.sh exited due to signal 15
Jun 30 17:00:17 bogon Keepalived_vrrp[16831]: VRRP_Script(chk_http_port) succeeded
Jun 30 17:00:18 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) Changing effective priority from 100 to 102
Jun 30 17:00:18 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Changing effective priority from 90 to 92
Jun 30 17:00:18 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Transition to MASTER STATE
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Entering MASTER STATE
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) setting protocol VIPs.
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.60
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:19 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.50
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:21 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.60
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:24 bogon Keepalived_vrrp[16831]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:28 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Received advert with higher priority 100, ours 92
Jun 30 17:00:28 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) Entering BACKUP STATE
Jun 30 17:00:28 bogon Keepalived_vrrp[16831]: VRRP_Instance(VI_2) removing protocol VIPs.

可以看到 对VRRP_Instance(VI_1)而言,192.168.88.129为MASTER;对VRRP_Instance(VI_2)而言,192.168.88.129为BACKUP


192.168.88.130上keepalived日志信息如下:

Jun 30 17:00:13 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) Transition to MASTER STATE
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) Entering MASTER STATE
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) setting protocol VIPs.
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.50
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:14 bogon Keepalived_vrrp[5752]: Sending gratuitous ARP on ens33 for 192.168.88.50
Jun 30 17:00:15 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) Received advert with higher priority 100, ours 92
Jun 30 17:00:15 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun 30 17:00:15 bogon Keepalived_vrrp[5752]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 17:00:26 bogon Keepalived[5749]: Stopping
Jun 30 17:00:26 bogon Keepalived_healthcheckers[5751]: Stopped
Jun 30 17:00:27 bogon Keepalived_vrrp[5752]: Stopped
Jun 30 17:00:27 bogon Keepalived[5749]: Stopped Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Jun 30 17:00:27 bogon Keepalived[19665]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2
Jun 30 17:00:27 bogon Keepalived[19665]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:27 bogon Keepalived[19666]: Starting Healthcheck child process, pid=19668
Jun 30 17:00:27 bogon Keepalived[19666]: Starting VRRP child process, pid=19669
Jun 30 17:00:27 bogon Keepalived_healthcheckers[19668]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:27 bogon Keepalived_vrrp[19669]: Registering Kernel netlink reflector
Jun 30 17:00:27 bogon Keepalived_vrrp[19669]: Registering Kernel netlink command channel
Jun 30 17:00:27 bogon Keepalived_vrrp[19669]: Registering gratuitous ARP shared channel
Jun 30 17:00:27 bogon Keepalived_vrrp[19669]: Opening file '/etc/keepalived/keepalived.conf'.
Jun 30 17:00:27 bogon Keepalived_vrrp[19669]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_1) removing protocol VIPs.
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) removing protocol VIPs.
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: Using LinkWatch kernel netlink reflector...
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_1) Entering BACKUP STATE
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP_Script(chk_http_port) succeeded
Jun 30 17:00:28 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) Transition to MASTER STATE
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_1) Changing effective priority from 90 to 92
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) Changing effective priority from 100 to 102
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) Entering MASTER STATE
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) setting protocol VIPs.
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.60
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:29 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: VRRP_Instance(VI_2) Sending/queueing gratuitous ARPs on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60
Jun 30 17:00:34 bogon Keepalived_vrrp[19669]: Sending gratuitous ARP on ens33 for 192.168.88.60

可以看到 对VRRP_Instance(VI_1)而言,192.168.88.130为BACKUP;对VRRP_Instance(VI_2)而言,192.168.88.130为MASTER。


浏览器访问http://192.168.88.50/,192.168.88.129上的nginx会处理该请求;浏览器访问http://192.168.88.60/,192.168.88.130上的nginx会处理该请求 !


当任何一台服务器上面的keepalived服务停掉后,另外一台服务器上面的keepalived都会进入MASTER状态处理请求。如这里停掉192.168.88.129上的keepalived服务,则192.168.88.130上的keepalived服务的VRRP_Instance(VI_1)进入MASTER状态,并对http://192.168.88.50/ http://192.168.88.60/进行处理!


查看192.168.88.130此时ip 地址如下:

相关实践学习
每个IT人都想学的“Web应用上云经典架构”实战
本实验从Web应用上云这个最基本的、最普遍的需求出发,帮助IT从业者们通过“阿里云Web应用上云解决方案”,了解一个企业级Web应用上云的常见架构,了解如何构建一个高可用、可扩展的企业级应用架构。
目录
相关文章
|
2月前
|
应用服务中间件 Linux 网络安全
使用Nginx免费版与Keepalived实现高可用性High Availablity方案
本文介绍了如何使用Nginx免费版与Keepalived实现高可用性(HA)方案,涵盖环境搭建、Keepalived安装配置、版本升级及主从模式设置。通过虚拟机测试,结合CentOS与宝塔,详细说明VIP配置与服务启动流程,助你构建稳定可靠的Web高可用架构。
|
4月前
|
负载均衡 前端开发 JavaScript
LVS-DR模式、keepalived、Nginx与Tomcat合作,打造动静分离,高效负载均衡与高可用性
为了采用这样的架构,你需要对LVS-DR、Keepalived、Nginx与Tomcat有一定的理解和掌握,同时也需要投入一些时间去研究和配置,但是一旦你把它运行起来,你将会发现,这一切都是值得的。
172 11
|
Java 应用服务中间件 Shell
Nginx+Keepalived+Tomcat 实现Web高可用集群
Nginx+Keepalived+Tomcat 实现Web高可用集群
334 0
|
运维 负载均衡 监控
Nginx加Keepalived实现高可用
使用Nginx和Keepalived来实现高可用性的方案,对于确保关键服务的稳定性和可靠性来说是非常有效的。此配置涉及多个步骤,包括各个服务的安装、设置及测试,目标是在主服务器故障时能无缝切换,以确保服务的持续可用。正确的配置和充分的测试是实现高可用性的保证,这也要求管理员对这些工具和它们背后的原理有深入的了解。
405 1
|
26天前
|
编解码 应用服务中间件 Linux
centos配置nginx-rtmp实现ffmpeg转码rtsp为rtmp视频流
centos配置nginx-rtmp实现ffmpeg转码rtsp为rtmp视频流
101 1
|
4月前
|
应用服务中间件 Linux 网络安全
Centos 8.0中Nginx配置文件和https正书添加配置
这是一份Nginx配置文件,包含HTTP与HTTPS服务设置。主要功能如下:1) 将HTTP(80端口)请求重定向至HTTPS(443端口),增强安全性;2) 配置SSL证书,支持TLSv1.1至TLSv1.3协议;3) 使用uWSGI与后端应用通信(如Django);4) 静态文件托管路径设为`/root/code/static/`;5) 定制错误页面(404、50x)。适用于Web应用部署场景。
615 87
|
6天前
|
Ubuntu 安全 应用服务中间件
详细指南:配置Nginx服务器在Ubuntu平台上
以上步骤涵盖了基本流程:从软件包管理器获取 Ngnix, 设置系统服务, 调整UFW规则, 创建并激活服务器块(也称作虚拟主机), 并进行了初步优化与加固措施。这些操作都是建立在命令行界面上,并假设用户具有必要权限(通常是root用户)来执行这些命令。每个操作都有其特定原因:例如,设置开机启动确保了即使重启后也能自动运行 Ngnix;而编辑server block则定义了如何处理进入特定域名请求等等。
97 18
|
8天前
|
Ubuntu 安全 应用服务中间件
详细指南:配置Nginx服务器在Ubuntu平台上
以上步骤涵盖了基本流程:从软件包管理器获取 Ngnix, 设置系统服务, 调整UFW规则, 创建并激活服务器块(也称作虚拟主机), 并进行了初步优化与加固措施。这些操作都是建立在命令行界面上,并假设用户具有必要权限(通常是root用户)来执行这些命令。每个操作都有其特定原因:例如,设置开机启动确保了即使重启后也能自动运行 Ngnix;而编辑server block则定义了如何处理进入特定域名请求等等。
96 17
|
4月前
|
负载均衡 应用服务中间件 nginx
Nginx配置与命令
Nginx 是一款高性能的 HTTP 和反向代理服务器,其配置文件灵活且功能强大。本文介绍了 Nginx 配置的基础结构和常用指令,包括全局块、Events 块、HTTP 块及 Server 块的配置方法,以及静态资源服务、反向代理、负载均衡、HTTPS 和 URL 重写等功能实现。此外,还提供了常用的 Nginx 命令操作,如启动、停止、重载配置和日志管理等,帮助用户高效管理和优化服务器性能。
|
1月前
|
数据建模 应用服务中间件 PHP
配置nginx容器和php容器协同工作成功,使用ip加端口的方式进行通信
本示例演示如何通过Docker挂载同一宿主目录至Nginx与PHP容器,实现PHP项目运行环境配置。需注意PHP容器中监听地址修改为0.0.0.0:9000,并调整Nginx配置中fastcgi_pass指向正确的IP与端口。同时确保Nginx容器中/var/www/html权限正确,以避免访问问题。
配置nginx容器和php容器协同工作成功,使用ip加端口的方式进行通信