二、下载部署service-nodeport文件用于对外提供服务
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
修改service文件，指定一下nodePort，使用30080端口和30443端口作为nodePort。因为nodePort端口必须在30000以上，更改完如下所示：
apiVersion: v1kind: Servicemetadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec: type: NodePort # 指定端口类型 ports: - name: http port: 80 targetPort: 80 nodePort: 30080 # 新增30080端口，http访问时需要加上该端口 protocol: TCP - name: https port: 443 targetPort: 443 nodePort: 30443 # 新增30443端口来对外映射，https访问时需要加上该端口 protocol: TCP selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx
三、部署一个tomcat用于测试ingress转发功能
# cat tomcat-svc-deployment.yamlapiVersion: v1kind: Servicemetadata: name: tomcat namespace: ingress-nginxspec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 # 容器port port: 8080 # service port - name: ajp targetPort: 8009 port: 8009 ---apiVersion: apps/v1kind: Deploymentmetadata: name: tomcat-deploy namespace: ingress-nginxspec: replicas: 1 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat ports: - name: http containerPort: 8080
四、定义ingress策略
cat ingress-tomcat.yaml
#cat ingress-tomcat.yamlapiVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-tomcat namespace: ingress-nginx annotations: kubernetes.io/ingress.class: "nginx" # nginx.ingress.kubernetes.io/rewrite-target: / # 重写规则 # nginx.ingress.kubernetes.io/use-regex: "true" # 开启use-regex启动正则path匹配spec: rules: - host: tomcat.ingress.com # 对应的域名 http: paths: - path: # url上下文 backend: # 后向转发，到对应的 serviceName:servicePort serviceName: tomcat servicePort: 8080
五、本地做hosts解析
将tomcat.ingress.com域名在本地做hosts解析，解析的ip为ingress-controller这个pod所在的node机器外网地址，然后浏览器访问：http://tomcat.ingress.com:30080
成功访问 到tomcat界面表示deployment+nodeport方式的ingress已经成功
DaemonSet+HostNetwork+nodeSelector
更改 mandatory.yaml 文件中deployment部分：
修改参数如下：
kind: Deployment #修改为DaemonSet
replicas: 1 #注销此行，DaemonSet不需要此参数
hostNetwork: true #添加该字段让docker使用物理机网络，在物理机暴露服务端口(80)，注意物理机80端口提前不能被占用
dnsPolicy: ClusterFirstWithHostNet #使用hostNetwork后容器会使用物理机网络包括DNS，会无法解析内部service，使用此参数让容器使用K8S的DNS。当前测试没添加。
nodeSelector:isingress: "true" #添加节点标签
tolerations: 添加对指定节点容忍度。当前测试没添加
apiVersion: extensions/v1beta1kind: DaemonSetmetadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec:# 注释Replicas# replicas: 1 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: serviceAccountName: nginx-ingress-serviceaccount # 选择对应标签的node nodeSelector: isIngress: "true" # 使用hostNetwork暴露服务 hostNetwork: true containers: - name: nginx-ingress-controller image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.26.1
给节点打标签，让ingress-controller以ds的方式跑在这台机器上
[root@k8s-master ingrress-nginx]# kubectl label node 192.168.2.220 isIngress=true[root@k8s-master ingrress-nginx]# kubectl get node -l isIngress=trueNAME STATUS ROLES AGE VERSION192.168.2.220 Ready node 76d v1.15.0
用修改完成后的mandatory.yaml文件部署ingress-controller控制器
[root@k8s-master ingrress-nginx]# kubectl apply -f mandatory.yaml namespace/ingress-nginx createdconfigmap/nginx-configuration createdconfigmap/tcp-services createdconfigmap/udp-services createdserviceaccount/nginx-ingress-serviceaccount createdclusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole createdrole.rbac.authorization.k8s.io/nginx-ingress-role createdrolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding createdclusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding createddaemonset.extensions/nginx-ingress-controller created[root@k8s-master ingress-daemonset]# kubectl get ds -n ingress-nginxNAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGEnginx-ingress-controller 1 1 1 1 1 isIngress=true 6m8s[root@k8s-master ingress-daemonset]# kubectl get po -n ingress-nginx -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-ingress-controller-d52lg 1/1 Running 0 2m1s 192.168.2.220 192.168.2.220 <none> <none>
可以看到，nginx-controller的pod已经部署在在192.168.2.220这个节点上了暴露nginx-controller到192.168.2.220上看下本地端口
[root@k8s-node01 ~]# netstat -lntup | grep nginxtcp 0 0 127.0.0.1:10246 0.0.0.0:* LISTEN 95817/nginx: master tcp 0 0 127.0.0.1:10247 0.0.0.0:* LISTEN 95817/nginx: master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 95817/nginx: master tcp 0 0 0.0.0.0:8181 0.0.0.0:* LISTEN 95817/nginx: master tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 95817/nginx: master tcp 0 0 127.0.0.1:10245 0.0.0.0:* LISTEN 95782/nginx-ingress tcp6 0 0 :::10254 :::* LISTEN 95782/nginx-ingress tcp6 0 0 :::80 :::* LISTEN 95817/nginx: master tcp6 0 0 :::8181 :::* LISTEN 95817/nginx: master tcp6 0 0 :::443 :::* LISTEN 95817/nginx: master
由于配置了hostnetwork，nginx已经在node主机本地监听80/443/8181端口。其中8181是nginx-controller默认配置的一个default backend。这样，只要访问node主机有公网IP，就可以直接映射域名来对外网暴露服务了。如果要nginx高可用的话，可以在多个node上部署，并在前面再搭建一套LVS+keepalive做负载均衡。用hostnetwork的另一个好处是，如果lvs用DR模式的话，是不支持端口映射的，这时候如果用nodeport，暴露非标准的端口，管理起来会很麻烦。
配置ingress资源
创建一个tomcat的deployment应用用来测试ingress转发功能。
因为我们创建的ingress-controller采用的是hostnetwork模式，所以无需在创建nodePort形式的ingress-svc服务来把端口映射到节点主机上。即svc文件中不需要指定nodePort
# cat tomcat-svc-deployment.yamlapiVersion: v1kind: Servicemetadata: name: tomcat namespace: ingress-nginxspec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 port: 8080 - name: ajp targetPort: 8009 port: 8009---apiVersion: apps/v1kind: Deploymentmetadata: name: tomcat-deploy namespace: ingress-nginxspec: replicas: 1 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat ports: - name: httpd containerPort: 8080
定义ingress策略，通过Ingress把tomcat发布出去
# cat ingress-tomcat.yamlapiVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-tomcat namespace: ingress-nginx annotations: kubernetes.io/ingress.class: "nginx" # nginx.ingress.kubernetes.io/rewrite-target: / # 重写规则 # nginx.ingress.kubernetes.io/ssl-redirect: "true" # 设置是否强制跳转 # nginx.ingress.kubernetes.io/use-regex: "true" # 开启use-regex启动正则path匹配spec: rules: - host: tomcat.ingress.com http: paths: - path: backend: serviceName: tomcat # 指定需要绑定暴露的svc名称 servicePort: 8080
部署tomcat应用的svc和deployment，并且部署ingress规则
[root@k8s-master ingress-daemonset]# kubectl apply -f tomcat-svc-deployment.yaml[root@k8s-master ingress-daemonset]# kubectl apply -f ingress-tomcat.yam[root@k8s-master ingress-daemonset]# kubectl get ds,po -n ingress-nginxNAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGEdaemonset.extensions/nginx-ingress-controller 1 1 1 1 1 isIngress=true 10mNAME READY STATUS RESTARTS AGEpod/nginx-ingress-controller-d52lg 1/1 Running 0 5m53spod/tomcat-deploy-69d84cbf7c-qxnnd 1/1 Running 0 83s[root@k8s-master ingress-daemonset]# kubectl get ing -n ingress-nginxNAME HOSTS ADDRESS PORTS AGEingress-tomcat tomcat.ingress.com 80 37s
最后浏览器直接通过域名访问，不加任何端口。成功访问到tomcat界面表示
ingress+DaemonSet+nodeSelector方式部署成功
生产环境中，建议把ingress通过DaemonSet的方式部署集群中，而且该节点打上污点不允许业务pod进行调度，以避免业务应用与Ingress服务发生资源争抢。然后通过SLB把ingress节点主机添为后端服务器，进行流量转发
配置ingress-nginx使用https
ingress-nginx创建https方法：
https://blog.csdn.net/bbwangj/article/details/82940419
生成ingress-nginx的tls类型secret创建命令：
kubectl create secret tls ingress-https --key xxxx.key --cert xxxx.pem <-n namespace>
一、制作自签证书
如果是阿里云上购买的域名，则可以申请一个免费的证书与之绑定，然后把证书下载下来再用来创建
[root@k8s-master ingress-daemonset]# openssl genrsa -out tls.key 2048[root@k8s-master ingress-daemonset]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=HangZhou/L=ZheJiang/O=devops/CN=tomcat.ingress.com
这时候会生成两个文件：tls.crt和tls.key
第二步：利用这两个文件创建secret
[root@k8s-master ingress-daemonset]# kubectl create secret tls tomcat-ingress-https --cert=tls.crt --key=tls.key -n ingress-nginxsecret/tomcat-ingress-https created[root@k8s-master ingress-daemonset]# kubectl get secret -n ingress-nginxNAME TYPE DATA AGEdefault-token-8465p kubernetes.io/service-account-token 3 17hnginx-ingress-serviceaccount-token-k2j27 kubernetes.io/service-account-token 3 17htomcat-ingress-https kubernetes.io/tls 2 6s
第三步：修改ingress策略，发布暴露tomcat应用添加tls部分
apiVersion: extensions/v1beta1kind: Ingressmetadata: name: ingress-tomcat namespace: ingress-nginx annotations: kubernetes.io/ingress.class: "nginx"spec: tls: # 添加tls - hosts: - tomcat.ingress.com # 指定证书绑定的域名 secretName: tomcat-ingress-https # 指定刚才创建的secret名称 rules: - host: tomcat.ingress.com http: paths: - path: backend: serviceName: tomcat servicePort: 8080
更新原来ingress发布策略
[root@k8s-master ingress-daemonset]# kubectl apply -f ingress-tomcat.yaml ingress.extensions/ingress-tomcat configured
web浏览器使用https访问测试：https://tomcat.ingress.com。
由于因为自制证书，所以这个是正常的。如果自制证书不想看到不安全，可以设置浏览器信任该证书。正常访问，则表示部署https方式的ingress成功
