1.5、部署kube-nginx
master节点需要kube-nginx- 使用Nginx 4层透明代理功能实现k8s节点(master节点和nginx节点)高可用访问kube-apiserver
- 控制节点的kube-controller-manager、kube-scheduler是多实例部署，所以只要一个实例正常，就可以保证集群高可用
- 集群内的Pod使用k8s服务域名kubernetes访问kube-apiserver，kube-dns会自动解析多个kube-apiserver节点的IP，所以也是高可用的
- 在每个Nginx进程，后端对接多个apiserver实例，Nginx对他们做健康检查和负载均衡
1.5.0、下载nginx二进制文件
k8s-01:~ # cd /opt/k8s/packages/ k8s-01:/opt/k8s/packages # wget http://nginx.org/download/nginx-1.16.1.tar.gz k8s-01:/opt/k8s/packages # tar xf nginx-1.16.1.tar.gz
1.5.1、编译部署nginx
k8s-01:~ # cd /opt/k8s/packages/nginx-1.16.1/ k8s-01:/opt/k8s/packages/nginx-1.16.1 # ./configure --prefix=$(pwd)/nginx-prefix \ --with-stream \ --without-http \ --without-http_uwsgi_module && \ make && \ make install
--with-stream开启 4 层透明转发(TCP Proxy)功能--without-xxx关闭功能，这样生成的动态链接二进制程序依赖最小
1.5.2、配置nginx.conf
k8s-01:~ # cd /opt/k8s/conf/ k8s-01:/opt/k8s/conf # cat > kube-nginx.conf <<EOF worker_processes 1; events { worker_connections 1024; } stream { upstream backend { hash \$remote_addr consistent; server 192.168.72.39:6443 max_fails=3 fail_timeout=30s; server 192.168.72.40:6443 max_fails=3 fail_timeout=30s; server 192.168.72.41:6443 max_fails=3 fail_timeout=30s; } server { listen *:8443; proxy_connect_timeout 1s; proxy_pass backend; } } EOF
- 注：将ip地址改为
master节点的ip，以自己的环境为准
1.5.3、配置nginx为systemctl管理
k8s-01:~ # cd /opt/k8s/conf/ k8s-01:/opt/k8s/conf # cat > kube-nginx.service <<EOF [Unit] Description=kube-apiserver nginx proxy After=network.target After=network-online.target Wants=network-online.target [Service] Type=forking ExecStartPre=/opt/k8s/server/kube-nginx/sbin/nginx \ -c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \ -p /opt/k8s/server/kube-nginx -t ExecStart=/opt/k8s/server/kube-nginx/sbin/nginx \ -c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \ -p /opt/k8s/server/kube-nginx ExecReload=/opt/k8s/server/kube-nginx/sbin/nginx \ -c /opt/k8s/server/kube-nginx/conf/kube-nginx.conf \ -p /opt/k8s/server/kube-nginx -s reload PrivateTmp=true Restart=always RestartSec=5 StartLimitInterval=0 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF
1.5.4、分发nginx二进制文件和配置文件
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${NODE_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" ssh root@${host} "mkdir -p /opt/k8s/server/kube-nginx/{conf,logs,sbin}" scp /opt/k8s/packages/nginx-1.16.1/nginx-prefix/sbin/nginx ${host}:/opt/k8s/server/kube-nginx/sbin/ scp /opt/k8s/conf/kube-nginx.conf ${host}:/opt/k8s/server/kube-nginx/conf/ scp /opt/k8s/conf/kube-nginx.service ${host}:/etc/systemd/system/ done
1.5.5、启动kube-nginx服务
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${NODE_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" ssh root@${host} "systemctl daemon-reload && \ systemctl enable kube-nginx --now && \ systemctl status kube-nginx | grep Active" done
