Kerbernetes的StatefulSet与Operator

本文涉及的产品
服务治理 MSE Sentinel/OpenSergo,Agent数量 不受限
云原生网关 MSE Higress,422元/月
注册配置 MSE Nacos/ZooKeeper,118元/月
简介: 文章主要介绍了Kubernetes中StatefulSet的概念、与Operator的关系、应用场景、限制条件、Pod标识符、Pod管理策略、更新策略,并通过实战案例展示了如何使用StatefulSet部署NFS网络文件系统、etcd服务和zookeeper服务,以及如何基于Operator部署etcd。

作者:尹正杰
版权声明:原创作品,谢绝转载!否则将追究法律责任。

一.StatefulSet概述

1>.StatefulSet与Operator的关系

  管理一组有状态pod服务的部署(deployment)和扩展(scaling),并保证这些pod的顺序和唯一性。
    与部署一样,StatefulSet管理基于相同容器规范的pod。
    与部署不同,statefulSet为每个pod维护一个粘性标识。这些pod是从同一个规范创建的,但不能互换:每个pod都有一个peristent标识符,可以在任何重新调度中维护。

  事实上,使用StatefulSet可以很轻松的帮咱们完成部署,但是扩展的话需要我们自己编写封装扩容集群或者缩容集群脚本代码的程序,这个程序我们统称为Operator。而且我们编写的脚本还需要进行大量测试,否则出现任意一个bug,将会在线上带来不堪设想的后果。
 
  于是,有一家公司叫CoreOS的公司对StatefulSet进行二次封装开源了Operator SDK(Software Development Kit)接口,用户可以借助于Operator SDK的API比较轻松定制出Operator控制器。

  这意味着第三方的程序员再去开发云原生应用不是针对Kubernetes云原生API,而是针对CoreOS公司的Operator SDK。换句话说,Kubernetes的大本营被人开了一道后门,从后方侵入快速拿下一半的云原生领土,而这个领土的所属方叫CoreOS。而CoreOS已经是Redhat旗下的产品,而Redhat又是IBM旗下的产品。

  在国外,微软(Microsoft),亚马逊(Amazon)已经是云计算的领头羊了,而IBM作为一个老牌的基础设施公司虽说也有自己的云计算产品但影响力却远远不如前两者公司。因为大公司通常是很难接收革新的,它们有自己成熟的产品也有充分的市场份额,有一个其它产品时竞争对手时,它们通常把竞争对手视为眼中钉肉中刺,而不是拥抱和开发它们,IBM就错失了很多良机。IBM之所以愿意买RedHat,原因之一可能是因为红帽公司旗下的CoreOS公司。

  IBM公司地位发展也给咱们运维人员一定的启发,这就好比你已经轻车熟路的使用K8S,突然间又出现了一个颠覆性的产品的苗头之后,你一定不是第一个愿意接收它的人,会有一种抵触心理,此时若很多人抱着开放的心胸去学习了该产品,恍然之间觉得自己的技能要被淘汰了不得不去学习的时候,那些先行者早已在这个行业称为巨头了。

  目前大部分应用也有自己的Operator开源项目,比较火热的应用在Github有对应的开源地址。博主推荐阅读:https://github.com/operator-framework/awesome-operators。

2>.StatefulSet的应用场景

  statefulset对于需要以下一项或多项的应用程序很有价值:
    (1)有唯一固定的网络标识符;
    (2)固定唯一的持久存储;
    (3)有序优雅的部署和扩展;
    (4)有序右眼的删除和终止;
    (5)有序自动的执行滚动更新;

  StatefulSet的限制条件(Limitations):
    (1)各Pod用到的存储卷必须使用由StorageClass动态供给给或由管理事先创建好的PV;
    (2)删除StatefulSet或缩减其规模导致Pod被删除时不应该自动删除(这个需要我们定义pvc指定其回收策略为"Retain")其存储卷以确保数据安全;
    (3)StatefulSet控制器依赖于一个事先存在的Headless Serivce对象实现Pod对象的持久,唯一的标识符配置;次Headless Service需要由用户手动配置。

3>.StatefulSet的Pod 标识符(Identity)

  有序索引(Ordinal Index):
    对于一个有N个副本的StatefulSet,StatefulSet中的每个Pod将被分配一个整数序数,从0到N-1,在这个集合上是唯一的。

  稳定网络ID(Stabel Network ID):
    StatefulSet中的每个Pod都从StatefulSet的名称和Pod的序号派生其主机名。
    StatefulSet可以使用无头服务(Headless Service)来控制其Pods的域。此服务管理的域的任务格式为:"$(service name).$(namespace).svc.cluster.local",其中“cluster.local”是群集域。
    创建每个Pod时,它将获得一个匹配的DNS子域,其格式为:"$(podname).$(governing service domain)",其中governing service由StatefulSet上的serviceName字段定义。

4>.Pod管理策略(Management Policies)

  在Kubernetes 1.7及更高版本中,StatefulSet允许您放宽其排序保证,同时通过其.spec.podManagementPolicay字段保持其唯一性和标识保证。

  有序(OrderedReady)Pod管理(Management):
    OrderedReady pod management是statefulset的默认设置。换句话说,就是当一个Pod创建成功后再回去创建下一个Pod,即有序创建Pod。  
  并行的(Parallel)Pod管理(Management):
    并行Pod管理告诉StatefulSet控制器并行启动或终止所有Pod,不要等到Pod运行并准备就绪或完全终止后再启动或终止另一个Pod。

5>.更新策略(Update Strategies)

  在Kubernetes 1.7及更高版本中,StatefulSet的.spec.updateStrategy字段允许您配置和禁用StatefulSet中Pods的容器、标签、资源请求/限制和注释的自动滚动更新。

  ON Delete:
    由用户手动删除现有的Pod对象从而触发其更新过程。

  Rolling Updates:
    默认策略,它通过自动更新机制完成更新过程,启动更新过程时,它自动删除每个Pod对象并以新配置进行重建,更新顺序同删除StatefulSet时的逆向操作机制,一次删除并更新一个Pod对象。  
  
  可以通过指定.spec.update strategy.RollingUpdate.partition对RollingUpdate更新策略进行分区。  
  
  如果指定了分区,则当StatefulSet的.pec.template被更新时,序号大于或等于分区的所有pod都将被更新。  
  
  序号小于分区的所有pod都将不会更新,即使它们被删除,也将在以前的版本中重新创建。  

二.StatefulSet实战案例

1>.构建NFS网络文件系统

[root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
--> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Running transaction check
---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
--> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
---> Package quota.x86_64 1:4.01-19.el7 will be installed
--> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
--> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
--> Running transaction check
---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
--> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
--> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
==============================================================================================================================================================================================================================================================================
Installing:
 nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
Installing for dependencies:
 gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
 keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
 libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
 libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
 libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
 libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
 libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
 libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
 libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
 libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
 quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
 quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
 rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
 tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+14 Dependent packages)

Total download size: 1.3 M
Installed size: 3.6 M
Downloading packages:
(1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
(2/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
(3/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
(4/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
(5/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
(6/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
(7/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
(8/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
(9/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
(10/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                       |  26 kB  00:00:00     
(11/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
(12/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
(13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
(14/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
(15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         2.9 MB/s | 1.3 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
  Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
  Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
  Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
  Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
  Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
  Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
  Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
  Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
  Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
  Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
  Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
  Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
  Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
  Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
  Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
  Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
  Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
  Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
  Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
  Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
  Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
  Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
  Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
  Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
  Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
  Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
  Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 

Installed:
  nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           

Dependency Installed:
  gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
  libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      

Complete!
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils

[root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5}
mkdir: created directory ‘/yinzhengjie/data’
mkdir: created directory ‘/yinzhengjie/data/volume1’
mkdir: created directory ‘/yinzhengjie/data/volume2’
mkdir: created directory ‘/yinzhengjie/data/volume3’
mkdir: created directory ‘/yinzhengjie/data/volume4’
mkdir: created directory ‘/yinzhengjie/data/volume5’
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ -R
/yinzhengjie/data/:
total 0
drwxr-xr-x 2 root root 6 Feb 10 06:04 volume1
drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2
drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3
drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4
drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5

/yinzhengjie/data/volume1:
total 0

/yinzhengjie/data/volume2:
total 0

/yinzhengjie/data/volume3:
total 0

/yinzhengjie/data/volume4:
total 0

/yinzhengjie/data/volume5:
total 0
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5}

[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# cat /etc/exports
/yinzhengjie/data/volume1    172.200.0.0/21(rw,no_root_squash)
/yinzhengjie/data/volume2    172.200.0.0/21(rw,no_root_squash)
/yinzhengjie/data/volume3    172.200.0.0/21(rw,no_root_squash)
/yinzhengjie/data/volume4    172.200.0.0/21(rw,no_root_squash)
/yinzhengjie/data/volume5    172.200.0.0/21(rw,no_root_squash)
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/
total 0
drwxr-xr-x 2 polkitd root 22 Feb 10 06:40 volume1
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume2
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume3
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume4
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume5
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# exportfs -rav
exporting 172.200.0.0/21:/yinzhengjie/data/volume5
exporting 172.200.0.0/21:/yinzhengjie/data/volume4
exporting 172.200.0.0/21:/yinzhengjie/data/volume3
exporting 172.200.0.0/21:/yinzhengjie/data/volume2
exporting 172.200.0.0/21:/yinzhengjie/data/volume1
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports

[root@test110.yinzhengjie.org.cn ~]# systemctl status nfs
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# systemctl start nfs
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# systemctl enable nfs
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# systemctl status nfs
● nfs-server.service - NFS server and services
   Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
  Drop-In: /run/systemd/generator/nfs-server.service.d
           └─order-with-mounts.conf
   Active: active (exited) since Mon 2020-02-10 06:08:52 CST; 6s ago
 Main PID: 6116 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/nfs-server.service

Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Starting NFS server and services...
Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Started NFS server and services.
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# ss -ntl | grep 2049
LISTEN     0      64           *:2049                     *:*                  
LISTEN     0      64          :::2049                    :::*                  
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# iptables -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# iptables -vnL -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# systemctl start nfs

2>.在K8S集群的每一个节点安装nfs客户端驱动

[root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirror.bit.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
--> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Running transaction check
---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
--> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
---> Package quota.x86_64 1:4.01-19.el7 will be installed
--> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
--> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
--> Running transaction check
---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
--> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
--> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
==============================================================================================================================================================================================================================================================================
Installing:
 nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
Installing for dependencies:
 gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
 keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
 libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
 libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
 libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
 libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
 libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
 libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
 libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
 libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
 quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
 quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
 rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
 tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+14 Dependent packages)

Total download size: 1.3 M
Installed size: 3.6 M
Downloading packages:
(1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
(2/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
(3/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
(4/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
(5/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
(6/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
(7/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
(8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
(9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
(10/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
(11/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                            |  50 kB  00:00:00     
(12/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
(13/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
(14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
(15/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.4 MB/s | 1.3 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
  Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
  Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
  Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
  Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
  Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
  Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
  Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
  Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
  Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
  Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
  Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
  Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
  Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
  Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
  Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
  Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
  Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
  Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
  Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
  Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
  Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
  Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
  Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
  Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
  Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
  Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
  Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 

Installed:
  nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           

Dependency Installed:
  gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
  libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      

Complete!
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils

[root@node201.yinzhengjie.org.cn ~]#  yum -y install nfs-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
--> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Running transaction check
---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
--> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
---> Package quota.x86_64 1:4.01-19.el7 will be installed
--> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
--> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
--> Running transaction check
---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
--> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
--> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
==============================================================================================================================================================================================================================================================================
Installing:
 nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
Installing for dependencies:
 gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
 keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
 libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
 libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
 libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
 libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
 libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
 libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
 libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
 libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
 quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
 quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
 rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
 tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+14 Dependent packages)

Total download size: 1.3 M
Installed size: 3.6 M
Downloading packages:
(1/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
(2/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
(3/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
(4/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
(5/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
(6/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
(7/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
(8/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
(9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
(10/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                         |  28 kB  00:00:00     
(11/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
(12/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
(13/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
(14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
(15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         2.2 MB/s | 1.3 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
  Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
  Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
  Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
  Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
  Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
  Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
  Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
  Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
  Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
  Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
  Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
  Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
  Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
  Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
  Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
  Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
  Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
  Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
  Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
  Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
  Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
  Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
  Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
  Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
  Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
  Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
  Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 

Installed:
  nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           

Dependency Installed:
  gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
  libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      

Complete!
[root@node201.yinzhengjie.org.cn ~]# 

[root@node201.yinzhengjie.org.cn ~]# yum -y install nfs-utils

[root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirror.bit.edu.cn
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
--> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Running transaction check
---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
--> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
---> Package quota.x86_64 1:4.01-19.el7 will be installed
--> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
--> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
--> Running transaction check
---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
--> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
--> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
==============================================================================================================================================================================================================================================================================
Installing:
 nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
Installing for dependencies:
 gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
 keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
 libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
 libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
 libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
 libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
 libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
 libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
 libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
 libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
 quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
 quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
 rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
 tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+14 Dependent packages)

Total download size: 1.3 M
Installed size: 3.6 M
Downloading packages:
(1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
(2/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
(3/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
(4/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
(5/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
(6/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
(7/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
(8/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
(9/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
(10/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
(11/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                            |  89 kB  00:00:00     
(12/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
(13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
(14/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
(15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.9 MB/s | 1.3 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
  Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
  Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
  Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
  Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
  Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
  Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
  Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
  Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
  Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
  Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
  Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
  Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
  Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
  Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
  Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
  Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
  Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
  Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
  Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
  Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
  Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
  Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
  Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
  Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
  Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
  Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
  Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 

Installed:
  nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           

Dependency Installed:
  gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
  libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      

Complete!
[root@node202.yinzhengjie.org.cn ~]# 

[root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils

[root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirror.bit.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
--> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
--> Running transaction check
---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
--> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
--> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
---> Package quota.x86_64 1:4.01-19.el7 will be installed
--> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
--> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
--> Running transaction check
---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
--> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
--> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
--> Running transaction check
---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
==============================================================================================================================================================================================================================================================================
Installing:
 nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
Installing for dependencies:
 gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
 keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
 libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
 libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
 libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
 libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
 libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
 libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
 libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
 libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
 quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
 quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
 rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
 tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+14 Dependent packages)

Total download size: 1.3 M
Installed size: 3.6 M
Downloading packages:
(1/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
(2/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
(3/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
(4/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
(5/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
(6/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
(7/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
(8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
(9/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                               |  90 kB  00:00:00     
(10/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
(11/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                       |  26 kB  00:00:00     
(12/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                      | 8.9 kB  00:00:00     
(13/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
(14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
(15/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         2.6 MB/s | 1.3 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
  Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
  Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
  Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
  Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
  Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
  Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
  Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
  Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
  Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
  Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
  Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
  Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
  Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
  Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
  Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
  Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
  Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
  Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
  Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
  Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
  Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
  Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
  Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
  Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
  Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
  Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
  Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 

Installed:
  nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           

Dependency Installed:
  gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
  libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      

Complete!
[root@node203.yinzhengjie.org.cn ~]# 

[root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils

[root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt      #在K8S node节点测试NFS服务是否可以成功挂载
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# mount | grep mnt
172.200.1.110:/yinzhengjie/data/volume1 on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.200.1.200,local_lock=none,addr=172.200.1.110)
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# df -h | grep mnt
172.200.1.110:/yinzhengjie/data/volume1  1.6T  416M  1.6T   1% /mnt
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt      #在K8S node节点测试NFS服务是否可以成功挂载,即验证可用性。

3>.创建nfs对应的pv

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v3
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume3
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v4
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume4
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v5
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume5
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml
persistentvolume/pv-nfs-v3 created
persistentvolume/pv-nfs-v4 created
persistentvolume/pv-nfs-v5 created
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl  get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Available                                   12s
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Available                                   11s
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Available                                   11s
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl  get pv -o wide
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE   VOLUMEMODE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Available                                   16s   Filesystem
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Available                                   15s   Filesystem
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Available                                   15s   Filesystem
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide

4>.基于statefulset部署Pod

[root@master200.yinzhengjie.org.cn ~]# kubectl  get pv -o wide
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE   VOLUMEMODE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Available                                   16s   Filesystem
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Available                                   15s   Filesystem
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Available                                   15s   Filesystem
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: yinzhengjie-sts
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-sts-svc
  namespace: yinzhengjie-sts
  labels:
    app: myapp
    controller: mystatefulset
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: myapp-pod
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-demo
  namespace: yinzhengjie-sts
spec:
  selector:
    matchLabels:
      app: myapp-pod
      controller: mystatefulset
  serviceName: "myapp-sts-svc"
  replicas: 2
  template:
    metadata:
      namespace: yinzhengjie-sts
      labels:
        app: myapp-pod 
        controller: mystatefulset
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: myapp-pvc
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: myapp-pvc
      namespace: yinzhengjie-sts
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 2Gi
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml 
namespace/yinzhengjie-sts created
service/myapp-sts-svc created
statefulset.apps/statefulset-demo created
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/statefulset-demo.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-sts
NAME                     READY   STATUS    RESTARTS   AGE
pod/statefulset-demo-0   1/1     Running   0          56s
pod/statefulset-demo-1   1/1     Running   0          55s

NAME                    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/myapp-sts-svc   ClusterIP   None         <none>        80/TCP    56s

NAME                                READY   AGE
statefulset.apps/statefulset-demo   2/2     56s
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-sts

[root@master200.yinzhengjie.org.cn ~]# kubectl get pvc -n yinzhengjie-sts
NAME                           STATUS   VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myapp-pvc-statefulset-demo-0   Bound    pv-nfs-v5   5Gi        RWO,ROX,RWX                   73s
myapp-pvc-statefulset-demo-1   Bound    pv-nfs-v3   5Gi        RWO,ROX,RWX                   72s
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get pvc -n yinzhengjie-sts

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                          STORAGECLASS   REASON   AGE     VOLUMEMODE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-1                           6m34s   Filesystem
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Available                                                                          6m33s   Filesystem
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-0                           6m33s   Filesystem
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv -o wide

[root@master200.yinzhengjie.org.cn ~]# kubectl get svc -n yinzhengjie-sts
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
myapp-sts-svc   ClusterIP   None         <none>        80/TCP    4m40s
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get svc -n yinzhengjie-sts

[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts -o wide
NAME                 READY   STATUS    RESTARTS   AGE     IP            NODE                         NOMINATED NODE   READINESS GATES
statefulset-demo-0   1/1     Running   0          5m21s   10.244.1.35   node201.yinzhengjie.org.cn   <none>           <none>
statefulset-demo-1   1/1     Running   0          5m20s   10.244.3.25   node203.yinzhengjie.org.cn   <none>           <none>
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts -o wide

5>.扩容Pod的副本数

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                          STORAGECLASS   REASON   AGE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-1                           18m
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Available                                                                          18m
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-0                           18m
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts 
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE                         NOMINATED NODE   READINESS GATES
pod/statefulset-demo-0   1/1     Running   0          13m   10.244.1.35   node201.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-1   1/1     Running   0          13m   10.244.3.25   node203.yinzhengjie.org.cn   <none>           <none>

NAME                    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/myapp-sts-svc   ClusterIP   None         <none>        80/TCP    13m   app=myapp-pod

NAME                                READY   AGE   CONTAINERS   IMAGES
statefulset.apps/statefulset-demo   2/2     13m   myapp        ikubernetes/myapp:v1
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=3 -n yinzhengjie-sts
statefulset.apps/statefulset-demo scaled
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                          STORAGECLASS   REASON   AGE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-1                           19m
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-2                           19m
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-0                           19m
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts 
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE                         NOMINATED NODE   READINESS GATES
pod/statefulset-demo-0   1/1     Running   0          15m   10.244.1.35   node201.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-1   1/1     Running   0          14m   10.244.3.25   node203.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-2   1/1     Running   0          14s   10.244.2.30   node202.yinzhengjie.org.cn   <none>           <none>

NAME                    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/myapp-sts-svc   ClusterIP   None         <none>        80/TCP    15m   app=myapp-pod

NAME                                READY   AGE   CONTAINERS   IMAGES
statefulset.apps/statefulset-demo   3/3     15m   myapp        ikubernetes/myapp:v1
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=3 -n yinzhengjie-sts

6>.升级Pod版本

[root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts 
NAME                     READY   STATUS    RESTARTS   AGE     IP            NODE                         NOMINATED NODE   READINESS GATES
pod/statefulset-demo-0   1/1     Running   0          19m     10.244.1.35   node201.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-1   1/1     Running   0          19m     10.244.3.25   node203.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-2   1/1     Running   0          4m29s   10.244.2.30   node202.yinzhengjie.org.cn   <none>           <none>

NAME                    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/myapp-sts-svc   ClusterIP   None         <none>        80/TCP    19m   app=myapp-pod

NAME                                READY   AGE   CONTAINERS   IMAGES
statefulset.apps/statefulset-demo   3/3     19m   myapp        ikubernetes/myapp:v1
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl set image sts statefulset-demo myapp=ikubernetes/myapp:v2 -n yinzhengjie-sts
statefulset.apps/statefulset-demo image updated
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -o wide -n yinzhengjie-sts 
NAME                     READY   STATUS    RESTARTS   AGE    IP            NODE                         NOMINATED NODE   READINESS GATES
pod/statefulset-demo-0   1/1     Running   0          69s    10.244.1.36   node201.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-1   1/1     Running   0          90s    10.244.3.26   node203.yinzhengjie.org.cn   <none>           <none>
pod/statefulset-demo-2   1/1     Running   0          103s   10.244.2.31   node202.yinzhengjie.org.cn   <none>           <none>

NAME                    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/myapp-sts-svc   ClusterIP   None         <none>        80/TCP    22m   app=myapp-pod

NAME                                READY   AGE   CONTAINERS   IMAGES
statefulset.apps/statefulset-demo   3/3     22m   myapp        ikubernetes/myapp:v2
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl set image sts statefulset-demo myapp=ikubernetes/myapp:v2 -n yinzhengjie-sts

[root@master200.yinzhengjie.org.cn ~]# kubectl describe sts statefulset-demo -n yinzhengjie-sts
Name:               statefulset-demo
Namespace:          yinzhengjie-sts
CreationTimestamp:  Wed, 12 Feb 2020 10:40:46 +0800
Selector:           app=myapp-pod,controller=mystatefulset
Labels:             <none>
Annotations:        kubectl.kubernetes.io/last-applied-configuration:
                      {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"statefulset-demo","namespace":"yinzhengjie-sts"},"spec":...
Replicas:           3 desired | 3 total
Update Strategy:    RollingUpdate
  Partition:        824644603528
Pods Status:        3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=myapp-pod
           controller=mystatefulset
  Containers:
   myapp:
    Image:        ikubernetes/myapp:v2
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>
    Mounts:
      /usr/share/nginx/html from myapp-pvc (rw)
  Volumes:  <none>
Volume Claims:
  Name:          myapp-pvc
  StorageClass:  
  Labels:        <none>
  Annotations:   <none>
  Capacity:      2Gi
  Access Modes:  [ReadWriteOnce]
Events:
  Type    Reason            Age                  From                    Message
  ----    ------            ----                 ----                    -------
  Normal  SuccessfulCreate  25m                  statefulset-controller  create Claim myapp-pvc-statefulset-demo-0 Pod statefulset-demo-0 in StatefulSet statefulset-demo success
  Normal  SuccessfulCreate  25m                  statefulset-controller  create Claim myapp-pvc-statefulset-demo-1 Pod statefulset-demo-1 in StatefulSet statefulset-demo success
  Normal  SuccessfulCreate  10m                  statefulset-controller  create Claim myapp-pvc-statefulset-demo-2 Pod statefulset-demo-2 in StatefulSet statefulset-demo success
  Normal  SuccessfulDelete  4m45s                statefulset-controller  delete Pod statefulset-demo-2 in StatefulSet statefulset-demo successful
  Normal  SuccessfulCreate  4m33s (x2 over 10m)  statefulset-controller  create Pod statefulset-demo-2 in StatefulSet statefulset-demo successful
  Normal  SuccessfulDelete  4m23s                statefulset-controller  delete Pod statefulset-demo-1 in StatefulSet statefulset-demo successful
  Normal  SuccessfulCreate  4m20s (x2 over 25m)  statefulset-controller  create Pod statefulset-demo-1 in StatefulSet statefulset-demo successful
  Normal  SuccessfulDelete  4m10s                statefulset-controller  delete Pod statefulset-demo-0 in StatefulSet statefulset-demo successful
  Normal  SuccessfulCreate  3m59s (x2 over 25m)  statefulset-controller  create Pod statefulset-demo-0 in StatefulSet statefulset-demo successful
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl describe sts statefulset-demo -n yinzhengjie-sts

7>.缩容Pod的副本数(缩容时pvc对应pv数据并不会被删除)

[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts
NAME                 READY   STATUS    RESTARTS   AGE
statefulset-demo-0   1/1     Running   0          19m
statefulset-demo-1   1/1     Running   0          19m
statefulset-demo-2   1/1     Running   0          5s
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl exec -it -n yinzhengjie-sts statefulset-demo-2 -- sh
/ # 
/ # cd /usr/share/nginx/html/        #进入到存储卷挂载路径
/usr/share/nginx/html # 
/usr/share/nginx/html # ls
/usr/share/nginx/html # 
/usr/share/nginx/html # echo "https://www.cnblogs.com/yinzhengjie/" > blog.txt      #在存储卷路径写入测试数据。
/usr/share/nginx/html # 
/usr/share/nginx/html # ls -l
total 4
-rw-r--r--    1 root     root            37 Feb 12  2020 blog.txt
/usr/share/nginx/html # 
/usr/share/nginx/html # exit
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

/usr/share/nginx/html # echo "https://www.cnblogs.com/yinzhengjie/" > blog.txt      #在存储卷路径写入测试数据。

[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts
NAME                 READY   STATUS    RESTARTS   AGE
statefulset-demo-0   1/1     Running   0          19m
statefulset-demo-1   1/1     Running   0          19m
statefulset-demo-2   1/1     Running   0          5s
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl exec -it -n yinzhengjie-sts statefulset-demo-2 -- sh
/ # 
/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # 
/usr/share/nginx/html # ls
/usr/share/nginx/html # 
/usr/share/nginx/html # echo "https://www.cnblogs.com/yinzhengjie/" > blog.txt
/usr/share/nginx/html # 
/usr/share/nginx/html # ls -l
total 4
-rw-r--r--    1 root     root            37 Feb 12  2020 blog.txt
/usr/share/nginx/html # 
/usr/share/nginx/html # exit
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=2 -n yinzhengjie-sts
statefulset.apps/statefulset-demo scaled
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-sts
NAME                 READY   STATUS    RESTARTS   AGE
statefulset-demo-0   1/1     Running   0          24m
statefulset-demo-1   1/1     Running   0          24m
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl scale sts statefulset-demo --replicas=2 -n yinzhengjie-sts      #对Pod的副本进行缩容

8>.删除statefulset

[root@master200.yinzhengjie.org.cn ~]# kubectl get sts -n yinzhengjie-sts
NAME               READY   AGE
statefulset-demo   3/3     55m
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl delete sts statefulset-demo  -n yinzhengjie-sts
statefulset.apps "statefulset-demo" deleted
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get sts -n yinzhengjie-sts
No resources found in yinzhengjie-sts namespace.
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl delete sts statefulset-demo -n yinzhengjie-sts

三.使用StatefulSet部署etcd服务

  StatefulSet(简称sts)可以做到简单的部署,扩容等功能,但是要涉及到比较复杂的功能比如数据备份等功能建议大家使用Operator。

1>.创建etcd的service资源

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml 
apiVersion: v1
kind: Service
metadata:
  name: etcd
  labels:
    app: etcd
  annotations:
    # Create endpoints also if the related pod isn't ready
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  ports:
  - port: 2379
    name: client
  - port: 2380
    name: peer
  clusterIP: None
  selector:
    app: etcd-member
---
apiVersion: v1
kind: Service
metadata:
  name: etcd-client
  labels:
    app: etcd
spec:
  ports:
  - name: etcd-client
    port: 2379
    protocol: TCP
    targetPort: 2379
  selector:
    app: etcd-member
  type: NodePort
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml 
service/etcd created
service/etcd-client created
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get svc 
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
etcd          ClusterIP   None             <none>        2379/TCP,2380/TCP   14s
etcd-client   NodePort    10.107.187.160   <none>        2379:32360/TCP      14s
kubernetes    ClusterIP   10.96.0.1        <none>        443/TCP             7d16h
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-services.yaml

[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get svc -o wide
NAME          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE     SELECTOR
etcd          ClusterIP   None             <none>        2379/TCP,2380/TCP   31s     app=etcd-member
etcd-client   NodePort    10.107.187.160   <none>        2379:32360/TCP      31s     app=etcd-member
kubernetes    ClusterIP   10.96.0.1        <none>        443/TCP             7d16h   <none>
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl get svc -o wide

2>.确认nfs有足够的空间及挂载目录并创建pv

[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# cat /etc/exports
/yinzhengjie/data/volume1    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume2    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume3    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume4    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume5    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume6    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume7    172.200.0.0/21(rw,sync,no_root_squash)
/yinzhengjie/data/volume8    172.200.0.0/21(rw,sync,no_root_squash)
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# vim /etc/exports

[root@test110.yinzhengjie.org.cn ~]# mkdir /yinzhengjie/data/volume{6,7,8}
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/
total 0
drwxr-xr-x 2 polkitd root 22 Feb 10 06:40 volume1
drwxr-xr-x 2 polkitd root  6 Feb 10 20:04 volume2
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume3
drwxr-xr-x 2 root    root 22 Feb 13 03:11 volume4
drwxr-xr-x 2 root    root  6 Feb 10 06:04 volume5
drwxr-xr-x 2 root    root  6 Feb 13 03:42 volume6
drwxr-xr-x 2 root    root  6 Feb 13 03:42 volume7
drwxr-xr-x 2 root    root  6 Feb 13 03:42 volume8
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# exportfs -rav
exporting 172.200.0.0/21:/yinzhengjie/data/volume8
exporting 172.200.0.0/21:/yinzhengjie/data/volume7
exporting 172.200.0.0/21:/yinzhengjie/data/volume6
exporting 172.200.0.0/21:/yinzhengjie/data/volume5
exporting 172.200.0.0/21:/yinzhengjie/data/volume4
exporting 172.200.0.0/21:/yinzhengjie/data/volume3
exporting 172.200.0.0/21:/yinzhengjie/data/volume2
exporting 172.200.0.0/21:/yinzhengjie/data/volume1
[root@test110.yinzhengjie.org.cn ~]# 
[root@test110.yinzhengjie.org.cn ~]# 

[root@test110.yinzhengjie.org.cn ~]# exportfs -rav

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v3
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume3
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v4
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume4
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v5
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume5
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v6
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume6
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v7
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume7
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs-v8
  labels:
    storsys: nfs
spec:
  accessModes: ["ReadWriteOnce","ReadOnlyMany","ReadWriteMany"]
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  nfs:
    server: 172.200.1.110
    path: /yinzhengjie/data/volume8
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml

[root@master200.yinzhengjie.org.cn ~]# kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                          STORAGECLASS   REASON   AGE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-1                           23m
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-2                           23m
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Bound    yinzhengjie-sts/myapp-pvc-statefulset-demo-0                           23m
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml 
persistentvolume/pv-nfs-v3 unchanged
persistentvolume/pv-nfs-v4 unchanged
persistentvolume/pv-nfs-v5 unchanged
persistentvolume/pv-nfs-v6 created
persistentvolume/pv-nfs-v7 created
persistentvolume/pv-nfs-v8 created
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                          STORAGECLASS   REASON   AGE
pv-nfs-v3   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-1                           3h52m
pv-nfs-v4   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-2                           3h52m
pv-nfs-v5   5Gi        RWO,ROX,RWX    Retain           Bound       yinzhengjie-sts/myapp-pvc-statefulset-demo-0                           3h52m
pv-nfs-v6   5Gi        RWO,ROX,RWX    Retain           Available                                                                          11s
pv-nfs-v7   5Gi        RWO,ROX,RWX    Retain           Available                                                                          11s
pv-nfs-v8   5Gi        RWO,ROX,RWX    Retain           Available                                                                          11s
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/statefulset/pv-nfs-demo.yaml

3>.创建etcd的实例StatefulSet配置文件

[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-statefulset.yaml 
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: etcd
  labels:
    app: etcd
spec:
  serviceName: etcd
  # changing replicas value will require a manual etcdctl member remove/add
  #   # command (remove before decreasing and add after increasing)
  replicas: 3
  selector:
    matchLabels:
      app: etcd-member
  template:
    metadata:
      name: etcd
      labels:
        app: etcd-member
    spec:
      containers:
      - name: etcd
        image: "quay.io/coreos/etcd:v3.2.16"
        ports:
        - containerPort: 2379
          name: client
        - containerPort: 2380
          name: peer
        env:
        - name: CLUSTER_SIZE
          value: "3"
        - name: SET_NAME
          value: "etcd"
        volumeMounts:
        - name: data
          mountPath: /var/run/etcd
        command:
          - "/bin/sh"
          - "-ecx"
          - |
            IP=$(hostname -i)
            PEERS=""
            for i in $(seq 0 $((${CLUSTER_SIZE} - 1))); do
                PEERS="${PEERS}${PEERS:+,}${SET_NAME}-${i}=http://${SET_NAME}-${i}.${SET_NAME}:2380"
            done
            # start etcd. If cluster is already initialized the `--initial-*` options will be ignored.
            exec etcd --name ${HOSTNAME} 
              --listen-peer-urls http://${IP}:2380 
              --listen-client-urls http://${IP}:2379,http://127.0.0.1:2379 
              --advertise-client-urls http://${HOSTNAME}.${SET_NAME}:2379 
              --initial-advertise-peer-urls http://${HOSTNAME}.${SET_NAME}:2380 
              --initial-cluster-token etcd-cluster-1 
              --initial-cluster ${PEERS} 
              --initial-cluster-state new 
              --data-dir /var/run/etcd/default.etcd
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: 1Gi
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/etcd-statefulset.yaml

四.使用StatefulSet部署zookeeper服务参考配置文件

[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/zk-sts.yaml 
apiVersion: v1
kind: Service
metadata:
  name: zk-hs
  labels:
    app: zk
spec:
  ports:
  - port: 2888
    name: server
  - port: 3888
    name: leader-election
  clusterIP: None
  selector:
    app: zk
---
apiVersion: v1
kind: Service
metadata:
  name: zk-cs
  labels:
    app: zk
spec:
  ports:
  - port: 2181
    name: client
  selector:
    app: zk
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: zk-pdb
spec:
  selector:
    matchLabels:
      app: zk
  maxUnavailable: 1
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: zk
spec:
  selector:
    matchLabels:
      app: zk
  serviceName: zk-hs
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  podManagementPolicy: Parallel
  template:
    metadata:
      labels:
        app: zk
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                    - zk-hs
              topologyKey: "kubernetes.io/hostname"
      containers:
      - name: kubernetes-zookeeper
        image: gcr.io/google-containers/kubernetes-zookeeper:1.0-3.4.10
        resources:
          requests:
            memory: "1Gi"
            cpu: "0.5"
        ports:
        - containerPort: 2181
          name: client
        - containerPort: 2888
          name: server
        - containerPort: 3888
          name: leader-election
        command:
        - sh
        - -c
        - "start-zookeeper 
          --servers=3 
          --data_dir=/var/lib/zookeeper/data 
          --data_log_dir=/var/lib/zookeeper/data/log 
          --conf_dir=/opt/zookeeper/conf 
          --client_port=2181 
          --election_port=3888 
          --server_port=2888 
          --tick_time=2000 
          --init_limit=10 
          --sync_limit=5 
          --heap=512M 
          --max_client_cnxns=60 
          --snap_retain_count=3 
          --purge_interval=12 
          --max_session_timeout=40000 
          --min_session_timeout=4000 
          --log_level=INFO"
        readinessProbe:
          exec:
            command:
            - sh
            - -c
            - "zookeeper-ready 2181"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        livenessProbe:
          exec:
            command:
            - sh
            - -c
            - "zookeeper-ready 2181"
          initialDelaySeconds: 10
          timeoutSeconds: 5
        volumeMounts:
        - name: data
          mountPath: /var/lib/zookeeper
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: gluster-dynamic
      resources:
        requests:
          storage: 5Gi
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 

[root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/statefulset/zk-sts.yaml

五.基于Operator部署etcd

  博主推荐阅读:
    https://github.com/coreos/etcd-operator
    https://github.com/coreos/etcd-operator/blob/master/example/deployment.yaml
    https://github.com/coreos/etcd-operator/blob/master/example/example-etcd-cluster.yaml

相关实践学习
通过Ingress进行灰度发布
本场景您将运行一个简单的应用,部署一个新的应用用于新的发布,并通过Ingress能力实现灰度发布。
容器应用与集群管理
欢迎来到《容器应用与集群管理》课程,本课程是“云原生容器Clouder认证“系列中的第二阶段。课程将向您介绍与容器集群相关的概念和技术,这些概念和技术可以帮助您了解阿里云容器服务ACK/ACK Serverless的使用。同时,本课程也会向您介绍可以采取的工具、方法和可操作步骤,以帮助您了解如何基于容器服务ACK Serverless构建和管理企业级应用。 学习完本课程后,您将能够: 掌握容器集群、容器编排的基本概念 掌握Kubernetes的基础概念及核心思想 掌握阿里云容器服务ACK/ACK Serverless概念及使用方法 基于容器服务ACK Serverless搭建和管理企业级网站应用
目录
相关文章
|
12月前
|
存储 Kubernetes 网络协议
【K8S系列】深入解析StatefulSet(一)
【K8S系列】深入解析StatefulSet(一)
385 1
|
存储 应用服务中间件 调度
在StatefulSet中使用LocalVolume存储卷保持节点一致
StatefulSet是一种有状态服务,其存储卷的使用有多种方式: 使用共享存储,这时在模板中定义一个volume卷,可以给多个pod共享; 每个pod配置独立的存储卷,使用非共享存储(块存储)时需要这样配置,通过配置volumeClaimTemplates实现; 对于StatefulSet使...
4720 0
|
2月前
|
存储 Kubernetes 网络协议
在K8S中,Deployment和Statefulset有何区别?
在K8S中,Deployment和Statefulset有何区别?
|
2月前
|
存储 消息中间件 Kubernetes
在K8S中,deploy和Statefulset有何区别?
在K8S中,deploy和Statefulset有何区别?
|
Web App开发 应用服务中间件 nginx
Operator ‘==‘ cannot be applied to ‘int‘, ‘null‘
Operator ‘==‘ cannot be applied to ‘int‘, ‘null‘
|
10月前
|
存储 Kubernetes 索引
Statefulset部署应用
Statefulset部署应用
|
10月前
|
Kubernetes API 索引
Statefulset实战 2
Statefulset实战 2
|
10月前
|
Cloud Native Go Docker
Statefulset 实战 1
Statefulset 实战 1
|
10月前
|
Kubernetes Cloud Native 索引
Statefulset 实战 3
Statefulset 实战 3