一、StatefulSet概述
StatefulSet（有状态集），通常用于部署有状态的并且需要有序启动的应用程序- StatefulSet主要用于管理有状态应用程序的工作负载API对象，例如：
生产环境中部署ElasticSearch集群、Mongodb集群，以及需要持久化的RabbitMQ集群、Redis集群、Kafka集群、ZooKeeper集群等
- 而StatefuleSet创建的Pod一般都会使用Headless Service（无头服务）进行通信，与普通的Service的区别在于：
1、Headless不分配ClusterIP，也就是不使用ClusterIP，而是使用Endpoint进行通信，一般的格式为：
statefulSetName-{0..N-1}.serviceName.namespace.svc.cluster.local #注释 1.statefulSetName：statefulSet的名称 2.{0..N-1}：Pod所在的序号，从0开始到n-1 3.serviceName：Headless Service的名称 4.namespace：服务所在的命名空间 5.cluster.local：Cluster Domain集群域
2、Headless Service可以通过解析Service的DNS，返回所有Pod的地址和DNS==（StatefulSet部署的Pod才有DNS）==
3、普通的Service，只能通过解析Service的DNS返回Service的ClusterIP
二、StatefulSet注意事项
- StatefulSet一般应用于符合以下一个或多个要求的应用程序：
- 需要持久化数据
- 需要有序的、优雅的部署和扩展
- 需要有序的自动安装更新
注释：如果应用程序不需要任何稳定的标识符或者有序部署、删除、扩展等，应该使用无状态的控制器，例如Deployment、ReplicaSet，通常都会使用Deployment
三、创建StatefulSet
- 创建一个简单的
StatefulSet：
kind: StatefulSet：这个参数定义了一个名叫nginx的StatefulSet，replicas表示部署Pod的副本数，也就是数量，这里写的副本数是1
kind: Service：这个参数定义了一个名称为nginx的Headless Service，创建的Service格式为nginx-0.nginx.default.svc.cluster.local，因为没有指定命名空间，所以默认是部署在default
[root@master test]# cat nginx.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: nginx spec: serviceName: "nginx" selector: matchLabels: run: nginx replicas: 1 template: metadata: labels: run: nginx spec: containers: - name: nginx image: nginx:1.15.2 ports: - containerPort: 80 name: nginx --- apiVersion: v1 kind: Service metadata: name: nginx labels: run: nginx spec: ports: - port: 80 name: nginx clusterIP: None selector: run: nginx
- 查看对象资源
[root@master test]# kubectl apply -f nginx.yaml #创建 statefulset.apps/nginx created service/nginx created [root@master test]# kubectl get pods #查看pod，可以看到nginx-0，这个名称是有编号的 NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 8s [root@master test]# kubectl get svc -o wide #创建的是无头服务，所以这里ip是none NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 172.16.0.1 <none> 443/TCP 8d <none> nginx ClusterIP None <none> 80/TCP 37s run=nginx [root@master test]# kubectl get statefulsets.apps NAME READY AGE nginx 1/1 24s [root@master test]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-0 1/1 Running 0 95s 10.244.1.35 node <none> <none> [root@master test]# kubectl describe svc nginx Name: nginx Namespace: default Labels: run=nginx Annotations: Selector: run=nginx Type: ClusterIP IP: None Port: nginx 80/TCP TargetPort: 80/TCP Endpoints: 10.244.1.35:80 Session Affinity: None Events: <none> #访问endpoints地址，可以成功访问 [root@master test]# curl 10.244.1.35 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
四、StatefulSet的扩容和缩容
- 与Deployment差不多，StatefulSet同样可以通过更新
replicas参数扩容、缩容StatefulSet，也可以使用kubectlscale或者kubectlpatch来扩容、缩容一个StatefulSet
（1）扩容
- 在上面创建的statefulSet基础上，将pod的副本数扩容到5个
注意：扩容之前必须保证有创建完成的静态PV、动态PV以及emptyDir
[root@master test]# kubectl scale statefulset nginx --replicas=5 #修改副本数为5个 statefulset.apps/nginx scaled [root@master test]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 18m nginx-1 1/1 Running 0 1s nginx-2 0/1 ContainerCreating 0 0s [root@master test]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 18m nginx-1 1/1 Running 0 4s nginx-2 1/1 Running 0 3s nginx-3 0/1 ContainerCreating 0 1s [root@master test]# kubectl get pods #可以看到是依次创建的 NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 18m nginx-1 1/1 Running 0 7s nginx-2 1/1 Running 0 6s nginx-3 1/1 Running 0 4s nginx-4 0/1 ContainerCreating 0 2s [root@master test]# kubectl get pods #可以看到创建的pod有顺序 NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 18m nginx-1 1/1 Running 0 30s nginx-2 1/1 Running 0 29s nginx-3 1/1 Running 0 27s nginx-4 1/1 Running 0 25s [root@master test]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-0 1/1 Running 0 27m 10.244.1.35 node <none> <none> nginx-1 1/1 Running 0 9m13s 10.244.1.36 node <none> <none> nginx-2 1/1 Running 0 9m12s 10.244.1.37 node <none> <none> nginx-3 1/1 Running 0 9m10s 10.244.1.38 node <none> <none> nginx-4 1/1 Running 0 9m8s 10.244.1.39 node <none> <none> #可以 [root@master test]# nslookup nginx.default.svc.cluster.local 172.16.0.10 Server: 172.16.0.10 Address: 172.16.0.10#53 Name: nginx.default.svc.cluster.local Address: 10.244.1.37 Name: nginx.default.svc.cluster.local Address: 10.244.1.35 Name: nginx.default.svc.cluster.local Address: 10.244.1.38 Name: nginx.default.svc.cluster.local Address: 10.244.1.39 Name: nginx.default.svc.cluster.local Address: 10.244.1.36
（2）缩容
[root@master test]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-0 1/1 Running 0 48m 10.244.1.35 node <none> <none> nginx-1 1/1 Running 0 30m 10.244.1.36 node <none> <none> nginx-2 1/1 Running 0 30m 10.244.1.37 node <none> <none> nginx-3 1/1 Running 0 30m 10.244.1.38 node <none> <none> nginx-4 1/1 Running 0 30m 10.244.1.39 node <none> <none> #修改副本数为3 [root@master test]# kubectl patch statefulset nginx -p '{"spec":{"replicas":3}}' statefulset.apps/nginx patched [root@master test]# kubectl get pods -w #加-w参数，可以实时查看，可以发现正在删除，并且是以序号从大到小删除 NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 54m nginx-1 1/1 Running 0 35m nginx-2 1/1 Running 0 35m nginx-3 1/1 Running 0 50s nginx-4 0/1 Terminating 0 48s nginx-4 0/1 Terminating 0 57s nginx-4 0/1 Terminating 0 57s nginx-3 1/1 Terminating 0 59s nginx-3 0/1 Terminating 0 59s nginx-3 0/1 Terminating 0 69s nginx-3 0/1 Terminating 0 69s [root@master test]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-0 1/1 Running 0 52m 10.244.1.35 node <none> <none> nginx-1 1/1 Running 0 34m 10.244.1.36 node <none> <none> nginx-2 1/1 Running 0 34m 10.244.1.37 node <none> <none>
（3）更新策略
- 先来查看yaml文件，使用
kubectl get statefulset -o yaml
updateStrategy: rollingUpdate: partition: 0 type: RollingUpdate #这个就是更新策略
- 修改更新策略
kubectl patch statefulset web -p '{"spec":{"updateStrategy":{"type":"RollingUpdate"}}}' #type选
更新策略
On Delete策略：OnDelete更新策略实现了传统（1.7版本之前）的行为。当我们选择这个更新策略并修改StatefulSet的.spec.template字段时，StatefulSet控制器不会自动更新Pod，我们必须手动删除Pod才能使控制器创建新的Pod。
RollingUpdate策略：RollingUpdate（滚动更新）更新策略会更新一个StatefulSet中所有的Pod，采用与序号索引相反的顺序进行滚动更新。
注释：
使用RollingUpdate策略更新下一个Pod前，StatefulSet控制器会终止每一个Pod并等待它们变成Running和Ready状态。在当前顺序变成Running和Ready状态之前，StatefulSet控制器不会更新下一个Pod，但它仍然会重建任何在更新过程中发生故障的Pod，使用它们当前的版本。已经接收到请求的Pod将会被恢复为更新的版本，没有收到请求的Pod则会被恢复为之前的版本。
注释：在更新过程中可以使用kubectl rollout status sts/<name>来查看滚动更新的状态
（4）分段更新
- StatefulSet可以使用RollingUpdate更新策略的
partition参数来分段更新一个StatefulSet，分段更新将会使pod序号小于partition（分区）数字的Pod保持当前版本，只更新序号大于分区的Pod，相当于是灰度发布 - 下面来看案例
#更新配置，更新分区数字为3，意思是每次更新使，不更新序号小于3的 [root@master test]# kubectl patch statefulset nginx -p '{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":3}}}}' statefulset.apps/nginx patched #这里可以更新一下镜像，我这里下载了新的nginx镜像，nginx:1.21.0 [root@master test]# docker pull nginx:1.21.0 1.21.0: Pulling from library/nginx b4d181a07f80: Pull complete edb81c9bc1f5: Pull complete b21fed559b9f: Pull complete 03e6a2452751: Pull complete b82f7f888feb: Pull complete 5430e98eba64: Pull complete Digest: sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09db Status: Downloaded newer image for nginx:1.21.0 docker.io/library/nginx:1.21.0 #因为上面定的分区数字有点大，所以这里再扩容一下 [root@master test]kubectl patch statefulset nginx -p '{"spec":{"replicas":7}}' statefulset.apps/nginx patched #修改镜像为nginx:1.21.0 [root@master test]# kubectl patch statefulset nginx --type='json' -p '[{"op":"replace","path":"/spec/template/spec/containers/0/image","value":"nginx:1.21.0"}]' #因为我有一个node节点，pod都创建再node上了，所以去node查看，对应容器使用的镜像id，可以发现序号大于等于3的容器使用的镜像都是新镜像 [root@node ~]# docker ps -a | grep nginx ca0156f6b0e1 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0 00f167982405 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0 ef28b0660a24 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0 cea1be3e6055 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0 acbe6faf965c 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0 ffdc06a8961b registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0 697436878a91 4f380adfc10f "/docker-entrypoint.…" 2 minutes ago Up 2 minutes k8s_nginx_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0 fde7dc2cf8ef registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0 8a6130d190af c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-2_default_107adf0c-856f-4aaa-b7a0-53515ac84d59_0 1720f7571d11 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-2_default_107adf0c-856f-4aaa-b7a0-53515ac84d59_0 62d3bde4de36 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-1_default_3336ee35-efa0-4b76-922d-c93146853f3b_0 db81735a8307 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-1_default_3336ee35-efa0-4b76-922d-c93146853f3b_0 f652001cd901 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0 6d781716840d registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0 [root@node ~]# docker images | grep nginx nginx 1.21.0 4f380adfc10f 13 months ago 133MB nginx 1.15.2 c82521676580 3 years ago 109MB #修改分区数字再次查看 [root@master test]# kubectl patch statefulset nginx -p '{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":1}}}}' statefulset.apps/nginx patched #可以看到，1号和2号，这两个pod重启了 [root@master test]# kubectl get pods -w NAME READY STATUS RESTARTS AGE nginx-0 1/1 Running 0 120m nginx-1 1/1 Running 0 16s nginx-2 1/1 Running 0 21s nginx-3 1/1 Running 0 5m51s nginx-4 1/1 Running 0 5m49s nginx-5 1/1 Running 0 5m47s nginx-6 1/1 Running 0 5m45s #查看容器，发现容器的镜像也更新了 [root@node ~]# docker ps -a | grep nginx 1b3a6390da17 4f380adfc10f "/docker-entrypoint.…" About a minute ago Up About a minute k8s_nginx_nginx-1_default_1a858475-b5ec-4e11-a740-6e3f86e6a65d_0 340a0193dfbd registry.aliyuncs.com/google_containers/pause:3.2 "/pause" About a minute ago Up About a minute k8s_POD_nginx-1_default_1a858475-b5ec-4e11-a740-6e3f86e6a65d_0 72ad32af50e8 4f380adfc10f "/docker-entrypoint.…" About a minute ago Up About a minute k8s_nginx_nginx-2_default_71559ce2-20df-4591-9f6c-ccc16886e0f8_0 848c4c4c6264 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" About a minute ago Up About a minute k8s_POD_nginx-2_default_71559ce2-20df-4591-9f6c-ccc16886e0f8_0 ca0156f6b0e1 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0 00f167982405 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-6_default_73ff98fa-3780-4f56-b3d2-bbbffeefac60_0 ef28b0660a24 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0 cea1be3e6055 registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-5_default_99995c4f-2677-4d2e-8941-2b359f7dfa76_0 acbe6faf965c 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0 ffdc06a8961b registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-4_default_3e339390-ad93-42be-b773-24eb7df26efe_0 697436878a91 4f380adfc10f "/docker-entrypoint.…" 6 minutes ago Up 6 minutes k8s_nginx_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0 fde7dc2cf8ef registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 6 minutes ago Up 6 minutes k8s_POD_nginx-3_default_04189cd3-2247-41b1-aa3f-be3cf525f76b_0 f652001cd901 c82521676580 "nginx -g 'daemon of…" 2 hours ago Up 2 hours k8s_nginx_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0 6d781716840d registry.aliyuncs.com/google_containers/pause:3.2 "/pause" 2 hours ago Up 2 hours k8s_POD_nginx-0_default_ad7d3529-9d96-4c82-a142-252823ea2c78_0 [root@node ~]# docker images | grep nginx nginx 1.21.0 4f380adfc10f 13 months ago 133MB nginx 1.15.2 c82521676580 3 years ago 109MB
五、dns解析
#service名称，nginx [root@master test]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 172.16.0.1 <none> 443/TCP 8d <none> nginx ClusterIP None <none> 80/TCP 35m run=nginx #kube-des的ip地址，172.16.0.10 [root@master test]# kubectl get svc -o wide -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR grafana NodePort 172.16.195.186 <none> 3000:30931/TCP 6d5h app=grafana,component=core kube-dns ClusterIP 172.16.0.10 <none> 53/UDP,53/TCP,9153/TCP 8d k8s-app=kube-dns metrics-server ClusterIP 172.16.122.15 <none> 443/TCP 27h k8s-app=metrics-server node-exporter NodePort 172.16.201.35 <none> 9100:31672/TCP 6d22h k8s-app=node-exporter prometheus NodePort 172.16.176.125 <none> 9090:30003/TCP 6d21h app=prometheus #通过解析dns，可以获取到pod的所有ip [root@master test]# nslookup nginx.default.svc.cluster.local 172.16.0.10 Server: 172.16.0.10 Address: 172.16.0.10#53 Name: nginx.default.svc.cluster.local Address: 10.244.1.37 Name: nginx.default.svc.cluster.local Address: 10.244.1.35 Name: nginx.default.svc.cluster.local Address: 10.244.1.38 Name: nginx.default.svc.cluster.local Address: 10.244.1.39 Name: nginx.default.svc.cluster.local Address: 10.244.1.36
nslookup nginx.default.svc.cluster.local 172.16.0.10格式为：
nginx：service的名称
default：命名空间名称
svc.cluster.local：固定的
172.16.0.10：kube-dns的ip地址
六、删除
- 删除的命令都是一样的
#直接这么删除，只会删除statefulSet资源，无法把创建的service删除 kubectl delete statefulset nginx #而这种删除可以把此yaml文件创建的所有资源对象一次性删除，也就是级联删除 kubectl delete -f nginx.yaml
