- 18.6 负载均衡集群介绍
- 18.7 LVS介绍
- 18.8 LVS调度算法
- 18.9/18.10 LVS NAT模式搭建
- 扩展
- lvs 三种模式详解 http:
- lvs几种算法 http:
- 关于arp_ignore和 arp_announcehttp:
- lvs原理相关的 http:
# 18.6 负载均衡集群介绍
- 主流开源软件LVS、keepalived、haproxy、nginx等
- 其中LVS属于4层（网络OSI 7层模型），nginx属于7层，haproxy既可以认为是4层，也可以当做7层使用
- keepalived的负载均衡功能其实就是lvs
- lvs这种4层的负载均衡是可以分发除80外的其他端口通信的，比如MySQL的，而nginx仅仅支持http，https，mail，haproxy也支持MySQL这种
- 相比较来说，LVS这种4层的更稳定，能承受更多的请求，而nginx这种7层的更加灵活，能实现更多的个性化需求
# 18.7 LVS介绍
- LVS是由国人章文嵩开发，（开源）
- 流行度不亚于apache的httpd，基于TCP/IP做的路由和转发，稳定性和效率很高
- LVS最新版本基于Linux内核2.6，有好多年不更新了
- LVS有三种常见的模式：NAT、DR、IP Tunnel
- LVS架构中有一个核心角色叫做分发器（Load balance），它用来分发用户的请求，还有诸多处理用户请求的服务器（Real Server，简称rs）
- LVS NAT模式
- 这种模式借助iptables的nat表来实现
- 用户的请求到分发器后，通过预设的iptables规则，把请求的数据包转发到后端的rs上去
- rs需要设定网关为分发器的内网ip
- 用户请求的数据包和返回给用户的数据包全部经过分发器，所以分发器成为瓶颈
- 在nat模式中，只需要分发器有公网ip即可，所以比较节省公网ip资源
- 
RX packets 9208 bytes 6415236 (6.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11214 bytes 937882 (915.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.202.150 netmask 255.255.255.0 broadcast 192.168.202.255
ether 00:0c:29:2e:28:f2 txqueuelen 1000 (Ethernet)
ens37: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.142.147 netmask 255.255.255.0 broadcast 192.168.142.255
inet6 fe80::20c:29ff:fe2e:28fc prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:2e:28:fc txqueuelen 1000 (Ethernet)
RX packets 474 bytes 43996 (42.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 238 bytes 32037 (31.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 50 bytes 4276 (4.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 50 bytes 4276 (4.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@aming-01 ~]#
```
- 
RX packets 2300 bytes 188527 (184.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 985 bytes 105210 (102.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.202.152 netmask 255.255.255.0 broadcast 192.168.202.255
ether 00:0c:29:58:33:e6 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 84 bytes 6884 (6.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 84 bytes 6884 (6.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@aming-02 ~]#
```
- rs2 网关设置192.168.202.130
```
[root@aming-03 ~]# ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.202.133 netmask 255.255.255.0 broadcast 192.168.202.255
inet6 fe80::4500:6d42:8612:4e53 prefixlen 64 scopeid 0x20<link>
inet6 fe80::ecdd:28b7:612b:cb7 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:9c:2b:f0 txqueuelen 1000 (Ethernet)
RX packets 2019 bytes 173062 (169.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1969 bytes 150115 (146.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.202.153 netmask 255.255.255.0 broadcast 192.168.202.255
ether 00:0c:29:9c:2b:f0 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 337 bytes 29100 (28.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 337 bytes 29100 (28.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@aming-03 ~]#
```
- 然后3台机器 都需要关闭下防火墙
- aming-01
```
[root@aming-01 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 1769 packets, 147K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 7346 packets, 401K bytes)
pkts bytes target prot opt in out source destination
[root@aming-01 ~]#
```
- aming-02
```
[root@aming-02 ~]# systemctl stop firewalld
[root@aming-02 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@aming-02 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
[root@aming-02 ~]#
```
- aming-03
```
[root@aming-03 ~]# systemctl stop firewalld
[root@aming-03 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@aming-03 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
[root@aming-03 ~]#
```
- 按照iptables-services 包 对第三台机器
```
[root@aming-03 ~]# cd /etc/yum.repos.d/
[root@aming-03 yum.repos.d]# ls
CentOS-Base.repo CentOS-fasttrack.repo CentOS-Vault.repo
CentOS-CR.repo CentOS-Media.repo epel.repo
CentOS-Debuginfo.repo CentOS-Sources.repo epel-testing.repo
[root@aming-03 yum.repos.d]# mv epel.repo epel.repo.1
改名是因为epel.repos包是国外的资源，下载速度慢
[root@aming-03 yum.repos.d]# yum lish |grep iptables-service
没有该命令：lish。请使用 /usr/bin/yum --help
[root@aming-03 yum.repos.d]# yum list |grep iptables-service
iptables-services.x86_64 1.4.21-18.2.el7_4 updates
[root@aming-03 yum.repos.d]# yum install -y iptables-services
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.163.com
* extras: centos.ustc.edu.cn
* updates: mirrors.163.com
正在解决依赖关系
--> 正在检查事务
---> 软件包 iptables-services.x86_64.0.1.4.21-18.2.el7_4 将被 安装
--> 正在处理依赖关系 iptables = 1.4.21-18.2.el7_4，它被软件包 iptables-services-1.4.21-18.2.el7_4.x86_64 需要
--> 正在检查事务
---> 软件包 iptables.x86_64.0.1.4.21-17.el7 将被 升级
---> 软件包 iptables.x86_64.0.1.4.21-18.2.el7_4 将被 更新
--> 解决依赖关系完成
依赖关系解决
============================================================================================
Package 架构 版本 源 大小
============================================================================================
正在安装:
iptables-services x86_64 1.4.21-18.2.el7_4 updates 51 k
为依赖而更新:
iptables x86_64 1.4.21-18.2.el7_4 updates 428 k
事务概要
============================================================================================
安装 1 软件包
升级 ( 1 依赖软件包)
总下载量：479 k
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/2): iptables-services-1.4.21-18.2.el7_4.x86_64.rpm | 51 kB 00:00:00
(2/2): iptables-1.4.21-18.2.el7_4.x86_64.rpm | 428 kB 00:00:00
--------------------------------------------------------------------------------------------
总计 520 kB/s | 479 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在更新 : iptables-1.4.21-18.2.el7_4.x86_64 1/3
正在安装 : iptables-services-1.4.21-18.2.el7_4.x86_64 2/3
清理 : iptables-1.4.21-17.el7.x86_64 3/3
验证中 : iptables-services-1.4.21-18.2.el7_4.x86_64 1/3
验证中 : iptables-1.4.21-18.2.el7_4.x86_64 2/3
验证中 : iptables-1.4.21-17.el7.x86_64 3/3
已安装:
iptables-services.x86_64 0:1.4.21-18.2.el7_4
作为依赖被升级:
iptables.x86_64 0:1.4.21-18.2.el7_4
完毕！
[root@aming-03 yum.repos.d]#
```
- 对第二台机器按照iptables-series包
```
[root@aming-02 ~]# yum install -y iptables-services
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.btte.net
* epel: mirrors.ustc.edu.cn
* extras: mirrors.163.com
* updates: mirrors.163.com
正在解决依赖关系
--> 正在检查事务
---> 软件包 iptables-services.x86_64.0.1.4.21-18.2.el7_4 将被 安装
--> 正在处理依赖关系 iptables = 1.4.21-18.2.el7_4，它被软件包 iptables-services-1.4.21-18.2.el7_4.x86_64 需要
--> 正在检查事务
---> 软件包 iptables.x86_64.0.1.4.21-17.el7 将被 升级
---> 软件包 iptables.x86_64.0.1.4.21-18.2.el7_4 将被 更新
--> 解决依赖关系完成
依赖关系解决
=======================================================================================================
Package 架构 版本 源 大小
=======================================================================================================
正在安装:
iptables-services x86_64 1.4.21-18.2.el7_4 updates 51 k
为依赖而更新:
iptables x86_64 1.4.21-18.2.el7_4 updates 428 k
事务概要
=======================================================================================================
安装 1 软件包
升级 ( 1 依赖软件包)
总下载量：479 k
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/2): iptables-services-1.4.21-18.2.el7_4.x86_64.rpm | 51 kB 00:00:00
(2/2): iptables-1.4.21-18.2.el7_4.x86_64.rpm | 428 kB 00:00:03
-------------------------------------------------------------------------------------------------------
总计 124 kB/s | 479 kB 00:00:03
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在更新 : iptables-1.4.21-18.2.el7_4.x86_64 1/3
正在安装 : iptables-services-1.4.21-18.2.el7_4.x86_64 2/3
清理 : iptables-1.4.21-17.el7.x86_64 3/3
验证中 : iptables-services-1.4.21-18.2.el7_4.x86_64 1/3
验证中 : iptables-1.4.21-18.2.el7_4.x86_64 2/3
验证中 : iptables-1.4.21-17.el7.x86_64 3/3
已安装:
iptables-services.x86_64 0:1.4.21-18.2.el7_4
作为依赖被升级:
iptables.x86_64 0:1.4.21-18.2.el7_4
完毕！
[root@aming-02 ~]#
```
- 对第二台aming-02 ,先把epel.repo 改名 再重新下载就快了
```
[root@aming-02 ~]# cd /etc/yum.repos.d/
[root@aming-02 yum.repos.d]# ls
CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo epel-testing.repo
CentOS-CR.repo CentOS-fasttrack.repo CentOS-Sources.repo epel.repo
[root@aming-02 yum.repos.d]# mv epel.repo epel.repo.1
```
- 查看哪个包 都安装了哪些文件
```
[root@aming-02 yum.repos.d]# systemctl start iptables-services
Failed to start iptables-services.service: Unit not found.
[root@aming-02 yum.repos.d]# rpm -ql iptables-services
/etc/sysconfig/ip6tables
/etc/sysconfig/iptables
/usr/lib/systemd/system/ip6tables.service
/usr/lib/systemd/system/iptables.service
/usr/libexec/initscripts/legacy-actions/ip6tables
/usr/libexec/initscripts/legacy-actions/ip6tables/panic
/usr/libexec/initscripts/legacy-actions/ip6tables/save
/usr/libexec/initscripts/legacy-actions/iptables
/usr/libexec/initscripts/legacy-actions/iptables/panic
/usr/libexec/initscripts/legacy-actions/iptables/save
/usr/libexec/iptables
/usr/libexec/iptables/ip6tables.init
/usr/libexec/iptables/iptables.init
[root@aming-02 yum.repos.d]#
```
- 开启iptables服务 启动服务 systemctl start iptables
- 设置开机启动 systemctl enable iptables
```
[root@aming-02 yum.repos.d]# systemctl start iptables
[root@aming-02 yum.repos.d]# systemctl enable iptables
Created symlink from /etc/systemd/system/basic.target.wants/iptables.service to /usr/lib/systemd/system/iptables.service.
[root@aming-02 yum.repos.d]#
```
- 第三台也是
```
[root@aming-03 yum.repos.d]# systemctl start iptables
[root@aming-03 yum.repos.d]# systemctl enable iptables
Created symlink from /etc/systemd/system/basic.target.wants/iptables.service to /usr/lib/systemd/system/iptables.service.
[root@aming-03 yum.repos.d]#
```
- 查看表，是否使用了netfilter服务了
```
[root@aming-03 yum.repos.d]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
65 5144 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT 48 packets, 4984 bytes)
pkts bytes target prot opt in out source destination
[root@aming-03 yum.repos.d]#
```
- 清空表的规则，以便后续实验
```
[root@aming-03 yum.repos.d]# iptables -F
[root@aming-03 yum.repos.d]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ 确定 ]
[root@aming-03 yum.repos.d]#
```
- 以上步骤需要检查另外两台 rs 机器是否开启firewalld 服务，如果开启切换为netfilter服务
- 对第二台 aming-02 这样做
```
[root@aming-02 yum.repos.d]# cd
[root@aming-02 ~]# iptables -F
[root@aming-02 ~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ 确定 ]
[root@aming-02 ~]#
```
- 把三台机器额 selinux 都关闭，临时关闭 setenforce 0 永久关闭 vi /etc/selinux/config，把里面的SELINUX=disabled
- 设置网关，必须把rs1 rs2 设置为分发器的ip地址192.168.202.130,设置好后 这俩台机器局不能访问外网了
- rs1 aming-02
```
[root@aming-01 ~]# systemctl stop firewalld
[root@aming-01 ~]# systemctl disable firewalld
[root@aming-02 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens33
[root@aming-02 ~]# systemctl restart network.service
[root@aming-02 ~]#
```
- rs2 aming-03
```
[root@aming-03 yum.repos.d]# vi /etc/sysconfig/network-scripts/ifcfg-ens33
[root@aming-03 yum.repos.d]# systemctl restart network.service
[root@aming-03 yum.repos.d]#
```
- 看下第一台分发器上 ,防火墙都关闭了
```
[root@aming-01 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 1923 packets, 159K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 6713 packets, 387K bytes)
pkts bytes target prot opt in out source destination
[root@aming-01 ~]# getenforce
Permissive
[root@aming-01 ~]#
```
- 准备工作就到这里了
# 18.10 LVS NAT模式搭建(下）
- 在dir上安装ipvsadm 在分发器dir上，安装ipvsadm ，这个是实现 lvs 的一个重要的工具，缺少这个工具，将没有办法实现 lvs 的功能
- yum install -y ipvsdam
- 因为epel.repo的包是国外资源，所以需要改个名字，再去yum 下载包
```
[root@aming-01 ~]# cd /etc/yum.repos.d
[root@aming-01 yum.repos.d]# ls
CentOS-Base.repo CentOS-Debuginfo.repo CentOS-Media.repo CentOS-Vault.repo epel-testing.repo
CentOS-CR.repo CentOS-fasttrack.repo CentOS-Sources.repo epel.repo
[root@aming-01 yum.repos.d]# mv epel.repo epel.repo.1
[root@aming-01 yum.repos.d]# yum install -y ipvsadm
已加载插件：fastestmirror
Determining fastest mirrors
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
正在解决依赖关系
--> 正在检查事务
---> 软件包 ipvsadm.x86_64.0.1.27-7.el7 将被 安装
--> 解决依赖关系完成
依赖关系解决
======================================================================================================================
Package 架构 版本 源 大小
======================================================================================================================
正在安装:
ipvsadm x86_64 1.27-7.el7 base 45 k
事务概要
======================================================================================================================
安装 1 软件包
总下载量：45 k
安装大小：75 k
Downloading packages:
ipvsadm-1.27-7.el7.x86_64.rpm | 45 kB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在安装 : ipvsadm-1.27-7.el7.x86_64 1/1
验证中 : ipvsadm-1.27-7.el7.x86_64 1/1
已安装:
ipvsadm.x86_64 0:1.27-7.el7
完毕！
[root@aming-01 yum.repos.d]#
```
- 在dir上编写脚本，vim /usr/local/sbin/lvs_nat.sh
- 编写一个脚本，用脚本进行维护会比较方便，不用一条命令一条命令的进行操作
```
[root@aming-01 yum.repos.d]# vi /usr/local/sbin/lvs_nat.sh
#! /bin/bash
# director 服务器上开启路由转发功能
echo 1 > /proc/sys/net/ipv4/ip_forward
# 关闭icmp的重定向
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
# 注意区分网卡名字，阿铭的两个网卡分别为ens33和ens37
echo 0 > /proc/sys/net/ipv4/conf/ens33/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/ens37/send_redirects
# director 设置nat防火墙
iptables -t nat -F
iptables -t nat -X
iptables -t nat -A POSTROUTING -s 192.168.202.0/24 -j MASQUERADE
# director设置ipvsadm
IPVSADM='/usr/sbin/ipvsadm'
$IPVSADM -C
$IPVSADM -A -t 192.168.142.147:80 -s lc -p 3
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.132:80 -m -w 1
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.133:80 -m -w 1
~
~
~
:wq
[root@aming-01 yum.repos.d]# vi /usr/local/sbin/lvs_nat.sh
[root@aming-01 yum.repos.d]#
```
- 可以执行下,没有输出，就是没有错误，一般有错误信息会直接报错出来
```
[root@aming-01 yum.repos.d]# sh /usr/local/sbin/lvs_nat.sh
[root@aming-01 yum.repos.d]#
```
- 两台rs上都安装nginx
- 设置两台rs的主页，做一个区分，也就是说直接curl两台rs的ip时，得到不同的结果
- 浏览器里访问192.168.142.147，多访问几次看结果差异
- 先去第二台机器上看下有没有启动nginx 服务
```
[root@aming-02 ~]# ps aux |grep nginx
root 4280 0.0 0.0 112680 980 pts/0 S+ 20:49 0:00 grep --color=auto nginx
[root@aming-02 ~]# netstat -lntp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 911/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1659/master
tcp6 0 0 :::3306 :::* LISTEN 1294/mysqld
tcp6 0 0 :::111 :::* LISTEN 1/systemd
tcp6 0 0 :::22 :::* LISTEN 911/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1659/master
[root@aming-02 ~]# systemctl start nginx
[root@aming-02 ~]# !ps
ps aux |grep nginx
root 4295 0.0 0.2 122792 2080 ? Ss 20:49 0:00 nginx: master process /usr/sbin/ngin
nginx 4296 0.0 0.3 123224 3124 ? S 20:49 0:00 nginx: worker process
root 4298 0.0 0.0 112680 980 pts/0 S+ 20:49 0:00 grep --color=auto nginx
[root@aming-02 ~]# curl localhost
backup backup.
[root@aming-02 ~]#
[root@aming-02 ~]# vi /usr/share/nginx/index.html
[root@aming-02 ~]# vi /usr/share/nginx/html/index.html
aming02.
~
~
~
~
:wq
[root@aming-02 ~]# vi /usr/share/nginx/html/index.html
[root@aming-02 ~]#
[root@aming-02 ~]# curl localhost
aming02.
[root@aming-02 ~]#
```
- 第三台也是这样配置,开启nginx服务
```
[root@aming-03 yum.repos.d]# systemctl start nginx
[root@aming-03 yum.repos.d]# vi /usr/share/nginx/html/index.html
aming03.
~
~
~
~
:wq
[root@aming-03 yum.repos.d]# vi /usr/share/nginx/html/index.html
[root@aming-03 yum.repos.d]# curl localhost
aming03.
[root@aming-03 yum.repos.d]#
```
- 我们现在可以做测试了，直接用windows 浏览器去访问192.168.142.147
- 
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 1 packets, 328 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 1 packets, 328 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 1 packets, 328 bytes)
pkts bytes target prot opt in out source destination
2 152 MASQUERADE all -- * * 192.168.202.0/24 0.0.0.0/0
[root@aming-01 yum.repos.d]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@aming-01 yum.repos.d]#
```
- 说明刚刚我们执行的脚本没有成功
```
[root@aming-01 ~]# !sh
sh /usr/local/sbin/lvs_nat.sh
invalid timeout value `0' specified
Memory allocation problem
Memory allocation problem
[root@aming-01 ~]#
```
- invalid timeout value `0' specified 不能设置为0 ，那把-p去掉
```
[root@aming-01 ~]# !vi
vi /usr/local/sbin/lvs_nat.sh
#! /bin/bash
# director 服务器上开启路由转发功能
echo 1 > /proc/sys/net/ipv4/ip_forward
# 关闭icmp的重定向
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
# 注意区分网卡名字，阿铭的两个网卡分别为ens33和ens37
echo 0 > /proc/sys/net/ipv4/conf/ens33/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/ens37/send_redirects
# director 设置nat防火墙
iptables -t nat -F
iptables -t nat -X
iptables -t nat -A POSTROUTING -s 192.168.202.0/24 -j MASQUERADE
# director设置ipvsadm
IPVSADM='/usr/sbin/ipvsadm'
$IPVSADM -C
$IPVSADM -A -t 192.168.142.147:80 -s lc
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.132:80 -m -w 1
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.133:80 -m -w 1
~
:wq
[root@aming-01 ~]# !sh
sh /usr/local/sbin/lvs_nat.sh
[root@aming-01 ~]#
```
- 再来看看,ipvsadm -ln 出来了数据
```
[root@aming-01 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.142.147:80 lc
-> 192.168.202.132:80 Masq 1 0 0
-> 192.168.202.133:80 Masq 1 0 0
[root@aming-01 ~]#
[root@aming-01 ~]#
[root@aming-01 ~]#
```
- 再换成 $IPVSADM -A -t 192.168.142.147:80 -s rr,之前是lc
```
[root@aming-01 ~]# vi /usr/local/sbin/lvs_nat.sh
#! /bin/bash
# director 服务器上开启路由转发功能
echo 1 > /proc/sys/net/ipv4/ip_forward
# 关闭icmp的重定向
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
# 注意区分网卡名字，阿铭的两个网卡分别为ens33和ens37
echo 0 > /proc/sys/net/ipv4/conf/ens33/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/ens37/send_redirects
# director 设置nat防火墙
iptables -t nat -F
iptables -t nat -X
iptables -t nat -A POSTROUTING -s 192.168.202.0/24 -j MASQUERADE
# director设置ipvsadm
IPVSADM='/usr/sbin/ipvsadm'
$IPVSADM -C
$IPVSADM -A -t 192.168.142.147:80 -s rr
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.132:80 -m -w 1
$IPVSADM -a -t 192.168.142.147:80 -r 192.168.202.133:80 -m -w 1
~
:wq
```
- 再来重新执行下
```
[root@aming-01 ~]# vi /usr/local/sbin/lvs_nat.sh
[root@aming-01 ~]# !sh
sh /usr/local/sbin/lvs_nat.sh
[root@aming-01 ~]#
```
- 
