作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.Ingress概述
管理对集群中服务的外部访问的API对象，通常是HTTP。Ingress是允许入站连接访问群集服务的规则集合。
Ingress可以配置为提供外部可接收的url、负载平衡通信、SSL终端和基于名称的虚拟主机等功能。
Ingress其实也是一个控制器(Controller)，只不过它并不被"kube-controller-manager"打包管理,而且一般情况下Ingress作为Pod来运行。
Ingress仅是用于定义流量转发和调度的通用格式的配置信息，它们需要转换为特定的具有http协议转发和调度功能的应用程序(例如nginx,haproxy,traeik等)的配置文件，并由相应的应用程序生效相应的配置后完成流量转发。
此类能理解Ingress定义的配置信息，并可将其转换为自身配置的应用程序，即为Ingress Controller。
此类的控制区需要由Kubernetes管理员额外以Addons的形式部署为Pod资源对象,它们通过API Server获取Ingress的相关定义；
这与其他类型的控制器不同，它们通常作为"kube-controller-manager"二进制文件的一部分运行，并且通常作为集群创建的一部分自动启动;
选择最适合集群的入口控制器(ingress controller)实现，或者实现一个新的入口控制器(ingress controller)，Kubernetes目前支持并维护GCE和nginx控制器(https://github.com/kubernetes/ingress-nginx)。
Ingress自身不支持使用标签选择器挑选真正提供服务的Pod对象，因此，它需要由Service对象的辅助完成此类功能(如借助Service的标签选择器功能过滤出后端的Pod)。
Ingress自身不运行使用标签选择器挑选真正提供服务的Pod对象,它需要由Service对象的辅助完成此类功能。
Ingress Controller根据Igress定义的配置调度流量时,其报文将由Ingress Controller直接调度后直达Pod对象,而不再经由Service调度。
Ingress Controller也是Pod对象，它能够与各后端Pod直接进行通信。
Ingres官方文档:
https://kubernetes.io/docs/concepts/services-networking/ingress/
二.在K8S集群部署Ingress-nginx
1>.参考kubernetes的官方文档(https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md)
2>.部署ingress-nginx
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get namespace
NAME STATUS AGE
default Active 3d17h
kube-node-lease Active 3d17h
kube-public Active 3d17h
kube-system Active 3d17h
myservice Active 4h19m
testing Active 21h
testing2 Active 11h
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.28.0/deploy/static/mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get namespace
NAME STATUS AGE
default Active 3d17h
ingress-nginx Active 11s
kube-node-lease Active 3d17h
kube-public Active 3d17h
kube-system Active 3d17h
myservice Active 4h20m
testing Active 21h
testing2 Active 11h
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 47s 10.244.3.5 node203.yinzhengjie.org.cn <none> <none>
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.28.0/deploy/static/mandatory.yaml
3>.根据上一步创建的Ingress-nginx的label创建Service资源
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx --show-labels
NAME READY STATUS RESTARTS AGE LABELS
nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 9m56s app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=5556bd798f
[root@master200.yinzhengjie.org.cn ~]#
root@master200.yinzhengjie.org.cn ~]# kubectl get service -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-ingress NodePort 10.104.132.19 <none> 80:31080/TCP,443:31910/TCP 33s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
name: my-ingress
namespace: ingress-nginx
spec:
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
ports:
- name: http
port: 80
nodePort: 30080
- name: https
port: 443
nodePort: 30443
type: NodePort
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml
service/my-ingress configured
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-ingress NodePort 10.104.132.19 <none> 80:30080/TCP,443:30443/TCP 2m58s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# hostname -i #如下图所示，可以通过NodePort方式访问到Ingress-nginx则说明部署成功了，我们可以清晰的看到nginx的版本,接下来咱们就该配置Ingress-nginx了。
172.200.1.200
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
三.应用Ingress-nginx实战案例
1>.创建后端Pod及Service
[root@master200.yinzhengjie.org.cn ~]# kubectl get ns
NAME STATUS AGE
default Active 3d23h
ingress-nginx Active 5h4m
kube-node-lease Active 3d23h
kube-public Active 3d23h
kube-system Active 3d23h
myservice Active 9h
testing Active 26h
testing2 Active 16h
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl create namespace yinzhengjie-ns
namespace/yinzhengjie-ns created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ns
NAME STATUS AGE
default Active 3d23h
ingress-nginx Active 5h6m
kube-node-lease Active 3d23h
kube-public Active 3d23h
kube-system Active 3d23h
myservice Active 9h
testing Active 26h
testing2 Active 16h
yinzhengjie-ns Active 2s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl create namespace yinzhengjie-ns #创建专门存放后端Pod的名称空间
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: mynginx
namespace: yinzhengjie-ns
spec:
replicas: 2
selector:
matchLabels:
app: mynginx
rel: beta
template:
metadata:
namespace: yinzhengjie-ns
labels:
app: mynginx
rel: beta
spec:
containers:
- name: mynginx
image: nginx:1.14-alpine
---
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: yinzhengjie-ns
spec:
selector:
app: myapp
rel: beta
ports:
- name: http
port: 80
targetPort: 80
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml #编写Pod和Service资源配置文件
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml
deployment.apps/mynginx created
service/myapp created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-ns -o wide --show-labels
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS
myapp ClusterIP 10.102.191.103 <none> 80/TCP 114s app=myapp,rel=beta <none>
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-ns -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
mynginx-c49cd4658-ks45t 1/1 Running 0 2m10s 10.244.1.12 node201.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta
mynginx-c49cd4658-l2676 1/1 Running 0 2m10s 10.244.3.6 node203.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml #应用yaml文件并验证是否资源是否创建成功
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-ns -o wide --show-labels
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
pod/mynginx-c49cd4658-ks45t 1/1 Running 0 3m4s 10.244.1.12 node201.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta
pod/mynginx-c49cd4658-l2676 1/1 Running 0 3m4s 10.244.3.6 node203.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS
service/myapp ClusterIP 10.102.191.103 <none> 80/TCP 3m4s app=myapp,rel=beta <none>
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR LABELS
deployment.apps/mynginx 2/2 2 2 3m4s mynginx nginx:1.14-alpine app=mynginx,rel=beta <none>
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR LABELS
replicaset.apps/mynginx-c49cd4658 2 2 2 3m4s mynginx nginx:1.14-alpine app=mynginx,pod-template-hash=c49cd4658,rel=beta app=mynginx,pod-template-hash=c49cd4658,rel=beta
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-ns -o wide --show-labels #查看"yinzhengjie-ns"的名称空间下所有资源
2>.创建Ingress资源,资源注解可参考官网(https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md)
[root@master200.yinzhengjie.org.cn ~]# kubectl explain ingress
KIND: Ingress
VERSION: extensions/v1beta1
DESCRIPTION:
Ingress is a collection of rules that allow inbound connections to reach
the endpoints defined by a backend. An Ingress can be configured to give
services externally-reachable urls, load balance traffic, terminate SSL,
offer name based virtual hosting etc. DEPRECATED - This group version of
Ingress is deprecated by networking.k8s.io/v1beta1 Ingress. See the release
notes for more information.
FIELDS:
apiVersion <string>
APIVersion defines the versioned schema of this representation of an
object. Servers should convert recognized schemas to the latest internal
value, and may reject unrecognized values. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind <string>
Kind is a string value representing the REST resource this object
represents. Servers may infer this from the endpoint the client submits
requests to. Cannot be updated. In CamelCase. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata <Object>
Standard object's metadata. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
spec <Object>
Spec is the desired state of the Ingress. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
status <Object>
Status is the current state of the Ingress. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl explain ingress
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myingress
namespace: yinzhengjie-ns
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: www.yinzhengjie.org.cn
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: 80
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml #编写Ingress的配置文件
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml
ingress.extensions/myingress created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns
NAME HOSTS ADDRESS PORTS AGE
myingress www.yinzhengjie.org.cn 80 4m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml #创建Igress资源
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"myingress","namespace":"yinzhengjie-ns"},"spec":{"rules":[{"host":"www.yinzhengjie.org.
cn","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}]}} kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
creationTimestamp: "2020-02-08T12:14:22Z"
generation: 1
name: myingress
namespace: yinzhengjie-ns
resourceVersion: "260013"
selfLink: /apis/extensions/v1beta1/namespaces/yinzhengjie-ns/ingresses/myingress
uid: 1300a1ef-e059-411d-88e3-da1640477f3b
spec:
rules:
- host: www.yinzhengjie.org.cn
http:
paths:
- backend:
serviceName: myapp
servicePort: 80
path: /
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns -o yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns
Name: myingress
Namespace: yinzhengjie-ns
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
www.yinzhengjie.org.cn
/ myapp:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"myingress","namespace":"yinzhengjie-ns"},"
spec":{"rules":[{"host":"www.yinzhengjie.org.cn","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 9m16s nginx-ingress-controller Ingress yinzhengjie-ns/myingress
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns
3>.验证创建的Ingress资源
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 5h53m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh
/etc/nginx $
/etc/nginx $ ls -l
total 88
-rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf
-rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf.default
-rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params
-rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params.default
drwxr-xr-x 2 www-data www-data 68 Jan 14 17:04 geoip
-rw-r--r-- 1 www-data www-data 2837 Jan 14 17:09 koi-utf
-rw-r--r-- 1 www-data www-data 2223 Jan 14 17:09 koi-win
drwxr-xr-x 6 www-data www-data 267 Jan 28 11:13 lua
-rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types
-rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types.default
drwxr-xr-x 2 www-data www-data 53 Jan 14 17:09 modsecurity
drwxr-xr-x 2 www-data www-data 150 Jan 14 17:09 modules
-rw-r--r-- 1 www-data www-data 17391 Feb 8 11:21 nginx.conf
-rw-r--r-- 1 www-data www-data 2656 Jan 14 17:09 nginx.conf.default
-rw-r--r-- 1 www-data www-data 2 Jan 28 11:13 opentracing.json
drwxr-xr-x 6 www-data www-data 301 Jan 14 17:09 owasp-modsecurity-crs
-rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params
-rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params.default
drwxr-xr-x 2 www-data www-data 24 Jan 28 11:13 template
-rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params
-rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params.default
-rw-r--r-- 1 www-data www-data 3610 Jan 14 17:09 win-utf
/etc/nginx $
/etc/nginx $
/etc/nginx $ more nginx.conf #如下图所示，如果看到虚拟主机信息就ok啦~我们就可以在浏览器通过"www.yinzhengjie.org.cn:30080"或者"www.yinzhengjie.org.cn:300443"进行访问啦
四.使用Ingress配置SSL(TLS)
1>.自建证书
[root@master200.yinzhengjie.org.cn ~]# cd /yinzhengjie/data/k8s/manifests/basic/ingress/
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll
total 8
-rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml
-rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# openssl genrsa -out myapp.key 2048
Generating RSA private key, 2048 bit long modulus
.............................................................................+++
.........+++
e is 65537 (0x10001)
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll
total 12
-rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml
-rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml
-rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
/yinzhengjie/data/k8s/manifests/basic/ingress]# openssl genrsa -out myapp.key 2048 #生成私钥
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll
total 12
-rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml
-rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml
-rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# openssl req -new -x509 -key myapp.key -out myapp.crt -subj /C=CN/ST=Beijing/L=Beijing/O=Ops/CN=master.yinzhengjie.org.cn -days 3650
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll
total 16
-rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml
-rw-r--r-- 1 root root 1310 Feb 8 20:45 myapp.crt
-rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml
-rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
/yinzhengjie/data/k8s/manifests/basic/ingress]# openssl req -new -x509 -key myapp.key -out myapp.crt -subj /C=CN/ST=Beijing/L=Beijing/O=Ops/CN=master.yinzhengjie.org.cn -days 3650 #生成自签证书
2>.配置证书资源
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret -h
Create a secret using specified subcommand.
Available Commands:
docker-registry Create a secret for use with a Docker registry
generic Create a secret from a local file, directory or literal value
tls Create a TLS secret
Usage:
kubectl create secret [flags] [options]
Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret -h
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls -h
Create a TLS secret from the given public/private key pair.
The public/private key pair must exist before hand. The public key certificate must be .PEM encoded and match the given
private key.
Examples:
# Create a new TLS secret named tls-secret with the given key pair:
kubectl create secret tls tls-secret --cert=path/to/tls.cert --key=path/to/tls.key
Options:
--allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in
the template. Only applies to golang and jsonpath output formats.
--append-hash=false: Append a hash of the secret to its name.
--cert='': Path to PEM encoded public key certificate.
--dry-run=false: If true, only print the object that would be sent, without sending it.
--generator='secret-for-tls/v1': The name of the API generator to use.
--key='': Path to private key associated with given certificate.
-o, --output='': Output format. One of:
json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-file.
--save-config=false: If true, the configuration of current object will be saved in its annotation. Otherwise, the
annotation will be unchanged. This flag is useful when you want to perform kubectl apply on this object in the future.
--template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The
template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].
--validate=true: If true, use a schema to validate the input before sending it
Usage:
kubectl create secret tls NAME --cert=path/to/cert/file --key=path/to/key/file [--dry-run] [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls -h
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll
total 16
-rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml
-rw-r--r-- 1 root root 1310 Feb 8 20:45 myapp.crt
-rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml
-rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls yinzhengjie.org.cn-cert -n yinzhengjie-ns --cert=myapp.crt --key=myapp.key --dry-run
secret/yinzhengjie.org.cn-cert created (dry run)
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls yinzhengjie.org.cn-cert -n yinzhengjie-ns --cert=myapp.crt --key=myapp.key
secret/yinzhengjie.org.cn-cert created
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls yinzhengjie.org.cn-cert -n yinzhengjie-ns --cert=myapp.crt --key=myapp.key
[root@master200.yinzhengjie.org.cn ~]# kubectl describe secret yinzhengjie.org.cn-cert -n yinzhengjie-ns
Name: yinzhengjie.org.cn-cert
Namespace: yinzhengjie-ns
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.crt: 1310 bytes
tls.key: 1675 bytes
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe secret yinzhengjie.org.cn-cert -n yinzhengjie-ns
3>.创建Ingress资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-tls
namespace: yinzhengjie-ns
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- master.yinzhengjie.org.cn
secretName: yinzhengjie.org.cn-cert
rules:
- host: master.yinzhengjie.org.cn
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: 80
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml
ingress.extensions/myapp-tls created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns
NAME HOSTS ADDRESS PORTS AGE
myapp-tls master.yinzhengjie.org.cn 80, 443 19s
myingress www.yinzhengjie.org.cn 80 47m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns
NAME HOSTS ADDRESS PORTS AGE
myapp-tls master.yinzhengjie.org.cn 80, 443 117s
myingress www.yinzhengjie.org.cn 80 49m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns myapp-tls
Name: myapp-tls
Namespace: yinzhengjie-ns
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
yinzhengjie.org.cn-cert terminates master.yinzhengjie.org.cn
Rules:
Host Path Backends
---- ---- --------
master.yinzhengjie.org.cn
/ myapp:80 (<none>)
Annotations:
kubernetes.io/ingress.class: nginx
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"myapp-tls","namespace":"yinzhengjie-ns"},"spec":{"rules":[{"host":"master.yinzhengjie.org.c
n","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["master.yinzhengjie.org.cn"],"secretName":"yinzhengjie.org.cn-cert"}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 2m3s nginx-ingress-controller Ingress yinzhengjie-ns/myapp-tls
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns myapp-tls
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 7h27m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh
/etc/nginx $
/etc/nginx $ ls
fastcgi.conf fastcgi_params geoip koi-win mime.types modsecurity nginx.conf opentracing.json scgi_params template uwsgi_params.default
fastcgi.conf.default fastcgi_params.default koi-utf lua mime.types.default modules nginx.conf.default owasp-modsecurity-crs scgi_params.default uwsgi_params win-utf
/etc/nginx $
/etc/nginx $ more nginx.conf #如下图所示，依旧可以看到证书相关的配置则说明咱们的配置成功啦~
五.通过Ingress反向代理Tomcat实战案例
1>.部署tomcat的Pod及其Service资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml
apiVersion: v1
kind: Namespace
metadata:
name: yinzhengjie-eshop
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mytomcat
namespace: yinzhengjie-eshop
spec:
replicas: 2
selector:
matchLabels:
app: tomcat
rel: beta
template:
metadata:
namespace: yinzhengjie-eshop
labels:
app: tomcat
rel: beta
spec:
containers:
- name: mytomcat
image: tomcat-alpine
---
apiVersion: v1
kind: Service
metadata:
name: mytomcat
namespace: yinzhengjie-eshop
spec:
selector:
app: tomcat
rel: beta
ports:
- name: http
port: 8080
targetPort: 8080
- name: ajp
port: 8089
targetPort: 8089
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml
namespace/yinzhengjie-eshop created
deployment.apps/mytomcat created
service/mytomcat created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ns
NAME STATUS AGE
default Active 4d2h
ingress-nginx Active 8h
kube-node-lease Active 4d2h
kube-public Active 4d2h
kube-system Active 4d2h
myservice Active 12h
testing Active 29h
testing2 Active 20h
yinzhengjie-eshop Active 3m25s
yinzhengjie-ns Active 3h17m
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-eshop
NAME READY STATUS RESTARTS AGE
mytomcat-6876768796-45rqs 0/1 ImagePullBackOff 0 3m38s
mytomcat-6876768796-pjkwd 0/1 ImagePullBackOff 0 3m38s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-eshop -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mytomcat-6876768796-45rqs 0/1 ErrImagePull 0 3m53s 10.244.1.21 node201.yinzhengjie.org.cn <none> <none>
mytomcat-6876768796-pjkwd 0/1 ErrImagePull 0 3m53s 10.244.2.17 node202.yinzhengjie.org.cn <none> <none>
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-eshop -o wide
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-eshop -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
mytomcat ClusterIP 10.106.29.218 <none> 8080/TCP,8089/TCP 3m59s app=tomcat,rel=beta
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-eshop
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mytomcat ClusterIP 10.106.29.218 <none> 8080/TCP,8089/TCP 4m3s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-eshop -o wide
2>.创建Igress资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tomcat
namespace: yinzhengjie-eshop
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: mytomcat.yinzhengjie.org.cn
http:
paths:
- path: /
backend:
serviceName: mytomcat
servicePort: 8080
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml
ingress.extensions/tomcat created
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-eshop
NAME HOSTS ADDRESS PORTS AGE
tomcat mytomcat.yinzhengjie.org.cn 80 13s
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-eshop
Name: tomcat
Namespace: yinzhengjie-eshop
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
mytomcat.yinzhengjie.org.cn
/ mytomcat:8080 ()
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"tomcat","namespace":"yinzhengjie-eshop"},"
spec":{"rules":[{"host":"mytomcat.yinzhengjie.org.cn","http":{"paths":[{"backend":{"serviceName":"mytomcat","servicePort":8080},"path":"/"}]}}]}}
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 38s nginx-ingress-controller Ingress yinzhengjie-eshop/tomcat
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-eshop
3>.验证配置是否生效
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 8h
[root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh
/etc/nginx $
/etc/nginx $ ls -l
total 92
-rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf
-rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf.default
-rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params
-rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params.default
drwxr-xr-x 2 www-data www-data 68 Jan 14 17:04 geoip
-rw-r--r-- 1 www-data www-data 2837 Jan 14 17:09 koi-utf
-rw-r--r-- 1 www-data www-data 2223 Jan 14 17:09 koi-win
drwxr-xr-x 6 www-data www-data 267 Jan 28 11:13 lua
-rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types
-rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types.default
drwxr-xr-x 2 www-data www-data 53 Jan 14 17:09 modsecurity
drwxr-xr-x 2 www-data www-data 150 Jan 14 17:09 modules
-rw-r--r-- 1 www-data www-data 24546 Feb 8 14:09 nginx.conf
-rw-r--r-- 1 www-data www-data 2656 Jan 14 17:09 nginx.conf.default
-rw-r--r-- 1 www-data www-data 2 Jan 28 11:13 opentracing.json
drwxr-xr-x 6 www-data www-data 301 Jan 14 17:09 owasp-modsecurity-crs
-rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params
-rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params.default
drwxr-xr-x 2 www-data www-data 24 Jan 28 11:13 template
-rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params
-rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params.default
-rw-r--r-- 1 www-data www-data 3610 Jan 14 17:09 win-utf
/etc/nginx $
/etc/nginx $
/etc/nginx $ more nginx.conf
