CentOS 7.6使用kubeadm部署k8s 1.17.2测试集群实战篇

本文涉及的产品
云解析 DNS,旗舰版 1个月
容器服务 Serverless 版 ACK Serverless,317元额度 多规格
容器服务 Serverless 版 ACK Serverless,952元额度 多规格
简介: 该博客文章详细介绍了在CentOS 7.6操作系统上使用kubeadm工具部署kubernetes 1.17.2版本的测试集群的过程,包括主机环境准备、安装Docker、配置kubelet、初始化集群、添加节点、部署网络插件以及配置k8s node节点管理api server服务器。

作者:尹正杰
版权声明:原创作品,谢绝转载!否则将追究法律责任。

  kubernetes技术已经称为原生云技术的事实标准,它是目前基础软件领域最为热门的分布式调度和管理平台。于是,kubernetes也几乎成了时下开发工程师和运维工程师必备的技能之一。今天我们就来一起搭建一个Mini版本的kubernetes集群来简单体验一下吧。

一.主机基础环境准备

1>.测试环境说明

  测试使用的kubernetes集群可由一个master主机及一个以上(建议至少两个)node主机组成,这些主机可以是物理服务器,也可以是vmware,virtualbox或kvm等虚拟化平台上的虚拟机,甚至是公有云上的VPS主机。

  本测试环境将由master200,node201,node202,node203四个独立的主机组成,它们分别拥有4核心的CPU及4G的内存资源,操作系统均为"CentOS Linux release 7.6.1810 (Core)",域名为"yinzhengjie.org.cn",具体配置如下图所示。  

  此外,各主机需要预设的系统环境如下:  
    (1)借助NTP服务设置节点时间精确同步;  
    (2)通过DNS完成各节点的主机名解析,测试环境主机数量较少时也可以使用hosts文件进行;  
    (3)关闭各节点的iptables或firewalld服务,并确保它们被禁止随系统引导过程启动;  
    (4)各节点禁用Selinux(否则在运行容器时可以会遇到各种奇葩报错);  
    (5)各节点禁用所有的swap设备(生产环境中强烈建议禁用,虽说K8S也支持配置参数来启用swap但这样会降低集群性能,使用"swapoff -a"只是临时关闭交换分区使用,永久关闭需要编辑"/etc/fstab"文件将挂载swap哪一行前面加一个"#"进行注释);  
    (6)若要使用ipvs模型的proxy,各节点还需要载入ipvs相关的各模块;

2>.搭建内网的时间服务器(设定集群各节点时钟同步)

[root@master200.yinzhengjie.org.cn ~]# yum -y install chrony
Loaded plugins: fastestmirror
Determining fastest mirrors
 * base: mirror.bit.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
(1/2): extras/7/x86_64/primary_db                                                                                                                                                                                                                      | 159 kB  00:00:00     
(2/2): updates/7/x86_64/primary_db                                                                                                                                                                                                                     | 5.9 MB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package chrony.x86_64 0:3.4-1.el7 will be installed
--> Processing Dependency: libseccomp.so.2()(64bit) for package: chrony-3.4-1.el7.x86_64
--> Running transaction check
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                             Arch                                                            Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 chrony                                                              x86_64                                                          3.4-1.el7                                                            base                                                          251 k
Installing for dependencies:
 libseccomp                                                          x86_64                                                          2.3.1-3.el7                                                          base                                                           56 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 306 k
Installed size: 788 k
Downloading packages:
(1/2): chrony-3.4-1.el7.x86_64.rpm                                                                                                                                                                                                                     | 251 kB  00:00:00     
(2/2): libseccomp-2.3.1-3.el7.x86_64.rpm                                                                                                                                                                                                               |  56 kB  00:00:00     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         1.7 MB/s | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Installing : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 
  Verifying  : libseccomp-2.3.1-3.el7.x86_64                                                                                                                                                                                                                              1/2 
  Verifying  : chrony-3.4-1.el7.x86_64                                                                                                                                                                                                                                    2/2 

Installed:
  chrony.x86_64 0:3.4-1.el7                                                                                                                                                                                                                                                   

Dependency Installed:
  libseccomp.x86_64 0:2.3.1-3.el7                                                                                                                                                                                                                                             

Complete!
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum -y install chrony

[root@master200.yinzhengjie.org.cn ~]# cp /etc/chrony.conf /etc/chrony.conf-`date +%F`
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# sed -r -i 's@(^server)@#\1@g' /etc/chrony.conf
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# sed -r -i 's@#(allow) 192.168.0.0/16@\1 172.200.0.0/21@' /etc/chrony.conf
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# sed -r -i 's@#(local)@\1@' /etc/chrony.conf
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server master200.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 172.200.0.0/21
allow 127.0.0.0/8
local stratum 10
logdir /var/log/chrony
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# systemctl restart chronyd.service 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl status chronyd.service 
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2020-02-04 17:25:31 CST; 4s ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
  Process: 5658 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
  Process: 5654 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 5656 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─5656 /usr/sbin/chronyd

Feb 04 17:25:31 master200.yinzhengjie.org.cn systemd[1]: Starting NTP client/server...
Feb 04 17:25:31 master200.yinzhengjie.org.cn chronyd[5656]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG)
Feb 04 17:25:31 master200.yinzhengjie.org.cn systemd[1]: Started NTP client/server.
Feb 04 17:25:35 master200.yinzhengjie.org.cn chronyd[5656]: Selected source 172.200.1.200
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl enable chronyd.service 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep chronyd
chronyd.service                               enabled 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# systemctl restart chronyd.service

[root@node201.yinzhengjie.org.cn ~]# vim /etc/chrony.conf 
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf
server master200.yinzhengjie.org.cn iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
logdir /var/log/chrony
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# systemctl restart chronyd.service 
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# systemctl enable chronyd.service
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep chronyd
chronyd.service                               enabled 
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/chrony.conf        #各node节点也需要秀给chrony服务的配置文件,将时间服务器地址指向咱们自建的时间服务器

[root@node201.yinzhengjie.org.cn ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
^* master200.yinzhengjie.or>    11   6    37    48    +75ns[+2097ns] +/-   87ms
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# chronyc sources

[root@node201.yinzhengjie.org.cn ~]# chronyc sourcestats -v
210 Number of sources = 1
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. 
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
master200.yinzhengjie.or>   6   6   136     -0.002      0.471    -23ns  4950ns
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# chronyc sourcestats -v

3>.启用ipvs内核模块

[root@master200.yinzhengjie.org.cn ~]# vim /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
#
#********************************************************************
#Author:        yinzhengjie
#QQ:             1053419035
#Date:             2019-11-30
#URL:             http://www.cnblogs.com/yinzhengjie
#Description:         enable ipvs script
#Copyright notice:     original works, no reprint! Otherwise, legal liability will be investigated.
#********************************************************************

ipvs_mods_dir="/usr/lib/modules/$(uname -r)/kernel/net/netfilter/ipvs"

for mod in $(ls $ipvs_mods_dir | grep -o "^[^.]*");do
    /usr/sbin/modinfo -F filename $mod &> /dev/null
    if [ $? -eq 0 ];then
        /sbin/modprobe $mod
    fi
done
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# vim /etc/sysconfig/modules/ipvs.modules

[root@master200.yinzhengjie.org.cn ~]# lsmod | grep ip_vs
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# chmod +x /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# lsmod | grep ip_vs
ip_vs_wrr              12697  0 
ip_vs_wlc              12519  0 
ip_vs_sh               12688  0 
ip_vs_sed              12519  0 
ip_vs_rr               12600  0 
ip_vs_pe_sip           12740  0 
nf_conntrack_sip       33860  1 ip_vs_pe_sip
ip_vs_nq               12516  0 
ip_vs_lc               12516  0 
ip_vs_lblcr            12922  0 
ip_vs_lblc             12819  0 
ip_vs_ftp              13079  0 
nf_nat                 26787  1 ip_vs_ftp
ip_vs_dh               12688  0 
ip_vs                 145497  24 ip_vs_dh,ip_vs_lc,ip_vs_nq,ip_vs_rr,ip_vs_sh,ip_vs_ftp,ip_vs_sed,ip_vs_wlc,ip_vs_wrr,ip_vs_pe_sip,ip_vs_lblcr,ip_vs_lblc
nf_conntrack          133095  3 ip_vs,nf_nat,nf_conntrack_sip
libcrc32c              12644  4 xfs,ip_vs,nf_nat,nf_conntrack
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# chmod +x /etc/sysconfig/modules/ipvs.modules

[root@master200.yinzhengjie.org.cn ~]# ll /etc/sysconfig/modules/ipvs.modules 
-rwxr-xr-x 1 root root 680 Feb  4 10:08 /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node201.yinzhengjie.org.cn:/etc/sysconfig/modules/
root@node201.yinzhengjie.org.cn's password: 
ipvs.modules                                                                                                                                                                                                                                100%  680   521.4KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node201.yinzhengjie.org.cn:/etc/sysconfig/modules/

[root@master200.yinzhengjie.org.cn ~]# ll /etc/sysconfig/modules/ipvs.modules 
-rwxr-xr-x 1 root root 680 Feb  4 10:08 /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node202.yinzhengjie.org.cn:/etc/sysconfig/modules/
root@node202.yinzhengjie.org.cn's password: 
ipvs.modules                                                                                                                                                                                                                                100%  680   521.4KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node202.yinzhengjie.org.cn:/etc/sysconfig/modules/

[root@master200.yinzhengjie.org.cn ~]# ll /etc/sysconfig/modules/ipvs.modules 
-rwxr-xr-x 1 root root 680 Feb  4 10:08 /etc/sysconfig/modules/ipvs.modules
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node203.yinzhengjie.org.cn:/etc/sysconfig/modules/
root@node203.yinzhengjie.org.cn's password: 
ipvs.modules                                                                                                                                                                                                                                100%  680   521.4KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp -p /etc/sysconfig/modules/ipvs.modules node203.yinzhengjie.org.cn:/etc/sysconfig/modules/

二.安装docker并启动(每个节点都需要安装docker环境)

1>.下载docker阿里云的软件源文件

[root@master200.yinzhengjie.org.cn ~]# cd /etc/yum.repos.d/
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# 
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# ll
total 32
-rw-r--r--. 1 root root 1664 Nov 23  2018 CentOS-Base.repo
-rw-r--r--. 1 root root 1309 Nov 23  2018 CentOS-CR.repo
-rw-r--r--. 1 root root  649 Nov 23  2018 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root  314 Nov 23  2018 CentOS-fasttrack.repo
-rw-r--r--. 1 root root  630 Nov 23  2018 CentOS-Media.repo
-rw-r--r--. 1 root root 1331 Nov 23  2018 CentOS-Sources.repo
-rw-r--r--. 1 root root 5701 Nov 23  2018 CentOS-Vault.repo
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# 
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
--2020-02-04 10:44:27--  https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 61.240.147.118, 27.221.92.111, 61.240.147.114, ...
Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|61.240.147.118|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2640 (2.6K) [application/octet-stream]
Saving to: ‘docker-ce.repo’

100%[===================================================================================================================================================================================================================>] 2,640       --.-K/s   in 0s      

2020-02-04 10:44:27 (69.9 MB/s) - ‘docker-ce.repo’ saved [2640/2640]

[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# 
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# ll
total 36
-rw-r--r--. 1 root root 1664 Nov 23  2018 CentOS-Base.repo
-rw-r--r--. 1 root root 1309 Nov 23  2018 CentOS-CR.repo
-rw-r--r--. 1 root root  649 Nov 23  2018 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root  314 Nov 23  2018 CentOS-fasttrack.repo
-rw-r--r--. 1 root root  630 Nov 23  2018 CentOS-Media.repo
-rw-r--r--. 1 root root 1331 Nov 23  2018 CentOS-Sources.repo
-rw-r--r--. 1 root root 5701 Nov 23  2018 CentOS-Vault.repo
-rw-r--r--  1 root root 2640 Feb  3 16:23 docker-ce.repo
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# 
[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]#

[root@master200.yinzhengjie.org.cn /etc/yum.repos.d]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

2>.安装docker

[root@master200.yinzhengjie.org.cn ~]# yum -y install docker-ce
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
docker-ce-stable                                                                                                                                                                                                                  | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/x86_64/primary_db                                                                                                                                                                                         |  37 kB  00:00:00     
(2/2): docker-ce-stable/x86_64/updateinfo                                                                                                                                                                                         |   55 B  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.5-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: libcgroup for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.107-3.el7 will be installed
--> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.107-3.el7.noarch
---> Package containerd.io.x86_64 0:1.2.10-3.2.el7 will be installed
---> Package docker-ce-cli.x86_64 1:19.03.5-3.el7 will be installed
---> Package libcgroup.x86_64 0:0.41-21.el7 will be installed
--> Running transaction check
---> Package policycoreutils-python.x86_64 0:2.5-33.el7 will be installed
--> Processing Dependency: policycoreutils = 2.5-33.el7 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Running transaction check
---> Package audit-libs-python.x86_64 0:2.8.5-4.el7 will be installed
--> Processing Dependency: audit-libs(x86-64) = 2.8.5-4.el7 for package: audit-libs-python-2.8.5-4.el7.x86_64
---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed
---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed
---> Package policycoreutils.x86_64 0:2.5-29.el7 will be updated
---> Package policycoreutils.x86_64 0:2.5-33.el7 will be an update
---> Package python-IPy.noarch 0:0.75-6.el7 will be installed
---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed
--> Running transaction check
---> Package audit-libs.x86_64 0:2.8.4-4.el7 will be updated
--> Processing Dependency: audit-libs(x86-64) = 2.8.4-4.el7 for package: audit-2.8.4-4.el7.x86_64
---> Package audit-libs.x86_64 0:2.8.5-4.el7 will be an update
--> Running transaction check
---> Package audit.x86_64 0:2.8.4-4.el7 will be updated
---> Package audit.x86_64 0:2.8.5-4.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================================
 Package                                                            Arch                                               Version                                                        Repository                                                    Size
=========================================================================================================================================================================================================================================================
Installing:
 docker-ce                                                          x86_64                                             3:19.03.5-3.el7                                                docker-ce-stable                                              24 M
Installing for dependencies:
 audit-libs-python                                                  x86_64                                             2.8.5-4.el7                                                    base                                                          76 k
 checkpolicy                                                        x86_64                                             2.5-8.el7                                                      base                                                         295 k
 container-selinux                                                  noarch                                             2:2.107-3.el7                                                  extras                                                        39 k
 containerd.io                                                      x86_64                                             1.2.10-3.2.el7                                                 docker-ce-stable                                              23 M
 docker-ce-cli                                                      x86_64                                             1:19.03.5-3.el7                                                docker-ce-stable                                              39 M
 libcgroup                                                          x86_64                                             0.41-21.el7                                                    base                                                          66 k
 libsemanage-python                                                 x86_64                                             2.5-14.el7                                                     base                                                         113 k
 policycoreutils-python                                             x86_64                                             2.5-33.el7                                                     base                                                         457 k
 python-IPy                                                         noarch                                             0.75-6.el7                                                     base                                                          32 k
 setools-libs                                                       x86_64                                             3.3.8-4.el7                                                    base                                                         620 k
Updating for dependencies:
 audit                                                              x86_64                                             2.8.5-4.el7                                                    base                                                         256 k
 audit-libs                                                         x86_64                                             2.8.5-4.el7                                                    base                                                         102 k
 policycoreutils                                                    x86_64                                             2.5-33.el7                                                     base                                                         916 k

Transaction Summary
=========================================================================================================================================================================================================================================================
Install  1 Package  (+10 Dependent packages)
Upgrade             (  3 Dependent packages)

Total download size: 90 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/14): audit-libs-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                         | 102 kB  00:00:00     
(2/14): audit-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                              | 256 kB  00:00:00     
(3/14): audit-libs-python-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                  |  76 kB  00:00:00     
(4/14): checkpolicy-2.5-8.el7.x86_64.rpm                                                                                                                                                                                          | 295 kB  00:00:00     
(5/14): container-selinux-2.107-3.el7.noarch.rpm                                                                                                                                                                                  |  39 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/containerd.io-1.2.10-3.2.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY                                                             ] 3.6 MB/s |  23 MB  00:00:18 ETA 
Public key for containerd.io-1.2.10-3.2.el7.x86_64.rpm is not installed
(6/14): containerd.io-1.2.10-3.2.el7.x86_64.rpm                                                                                                                                                                                   |  23 MB  00:00:02     
(7/14): libcgroup-0.41-21.el7.x86_64.rpm                                                                                                                                                                                          |  66 kB  00:00:01     
(8/14): libsemanage-python-2.5-14.el7.x86_64.rpm                                                                                                                                                                                  | 113 kB  00:00:03     
(9/14): python-IPy-0.75-6.el7.noarch.rpm                                                                                                                                                                                          |  32 kB  00:00:00     
(10/14): docker-ce-cli-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                   |  39 MB  00:00:05     
(11/14): setools-libs-3.3.8-4.el7.x86_64.rpm                                                                                                                                                                                      | 620 kB  00:00:00     
(12/14): policycoreutils-2.5-33.el7.x86_64.rpm                                                                                                                                                                                    | 916 kB  00:00:03     
(13/14): policycoreutils-python-2.5-33.el7.x86_64.rpm                                                                                                                                                                             | 457 kB  00:00:03     
(14/14): docker-ce-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                       |  24 MB  00:00:09     
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                    9.4 MB/s |  90 MB  00:00:09     
Retrieving key from https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
 From       : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Updating   : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                    2/17 
  Installing : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                         3/17 
  Installing : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 4/17 
  Installing : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                      5/17 
  Installing : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                 6/17 
  Installing : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         7/17 
  Installing : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Installing : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 9/17 
  Installing : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                            10/17 
  Installing : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                              11/17 
setsebool:  SELinux is disabled.
  Installing : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 12/17 
  Installing : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                    13/17 
  Updating   : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                            14/17 
  Cleanup    : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Cleanup    : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Cleanup    : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 
  Verifying  : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Verifying  : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                             2/17 
  Verifying  : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                             3/17 
  Verifying  : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                     4/17 
  Verifying  : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 5/17 
  Verifying  : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 6/17 
  Verifying  : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                               7/17 
  Verifying  : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Verifying  : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         9/17 
  Verifying  : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                   10/17 
  Verifying  : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 11/17 
  Verifying  : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                12/17 
  Verifying  : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                     13/17 
  Verifying  : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                        14/17 
  Verifying  : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Verifying  : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Verifying  : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 

Installed:
  docker-ce.x86_64 3:19.03.5-3.el7                                                                                                                                                                                                                       

Dependency Installed:
  audit-libs-python.x86_64 0:2.8.5-4.el7    checkpolicy.x86_64 0:2.5-8.el7                container-selinux.noarch 2:2.107-3.el7    containerd.io.x86_64 0:1.2.10-3.2.el7    docker-ce-cli.x86_64 1:19.03.5-3.el7    libcgroup.x86_64 0:0.41-21.el7   
  libsemanage-python.x86_64 0:2.5-14.el7    policycoreutils-python.x86_64 0:2.5-33.el7    python-IPy.noarch 0:0.75-6.el7            setools-libs.x86_64 0:3.3.8-4.el7       

Dependency Updated:
  audit.x86_64 0:2.8.5-4.el7                                                    audit-libs.x86_64 0:2.8.5-4.el7                                                    policycoreutils.x86_64 0:2.5-33.el7                                                   

Complete!
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum -y install docker-ce

[root@node201.yinzhengjie.org.cn ~]# yum -y install docker-ce
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                  | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/x86_64/primary_db                                                                                                                                                                                         |  37 kB  00:00:00     
(2/2): docker-ce-stable/x86_64/updateinfo                                                                                                                                                                                         |   55 B  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.5-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: libcgroup for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.107-3.el7 will be installed
--> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.107-3.el7.noarch
---> Package containerd.io.x86_64 0:1.2.10-3.2.el7 will be installed
---> Package docker-ce-cli.x86_64 1:19.03.5-3.el7 will be installed
---> Package libcgroup.x86_64 0:0.41-21.el7 will be installed
--> Running transaction check
---> Package policycoreutils-python.x86_64 0:2.5-33.el7 will be installed
--> Processing Dependency: policycoreutils = 2.5-33.el7 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Running transaction check
---> Package audit-libs-python.x86_64 0:2.8.5-4.el7 will be installed
--> Processing Dependency: audit-libs(x86-64) = 2.8.5-4.el7 for package: audit-libs-python-2.8.5-4.el7.x86_64
---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed
---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed
---> Package policycoreutils.x86_64 0:2.5-29.el7 will be updated
---> Package policycoreutils.x86_64 0:2.5-33.el7 will be an update
---> Package python-IPy.noarch 0:0.75-6.el7 will be installed
---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed
--> Running transaction check
---> Package audit-libs.x86_64 0:2.8.4-4.el7 will be updated
--> Processing Dependency: audit-libs(x86-64) = 2.8.4-4.el7 for package: audit-2.8.4-4.el7.x86_64
---> Package audit-libs.x86_64 0:2.8.5-4.el7 will be an update
--> Running transaction check
---> Package audit.x86_64 0:2.8.4-4.el7 will be updated
---> Package audit.x86_64 0:2.8.5-4.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================================
 Package                                                            Arch                                               Version                                                        Repository                                                    Size
=========================================================================================================================================================================================================================================================
Installing:
 docker-ce                                                          x86_64                                             3:19.03.5-3.el7                                                docker-ce-stable                                              24 M
Installing for dependencies:
 audit-libs-python                                                  x86_64                                             2.8.5-4.el7                                                    base                                                          76 k
 checkpolicy                                                        x86_64                                             2.5-8.el7                                                      base                                                         295 k
 container-selinux                                                  noarch                                             2:2.107-3.el7                                                  extras                                                        39 k
 containerd.io                                                      x86_64                                             1.2.10-3.2.el7                                                 docker-ce-stable                                              23 M
 docker-ce-cli                                                      x86_64                                             1:19.03.5-3.el7                                                docker-ce-stable                                              39 M
 libcgroup                                                          x86_64                                             0.41-21.el7                                                    base                                                          66 k
 libsemanage-python                                                 x86_64                                             2.5-14.el7                                                     base                                                         113 k
 policycoreutils-python                                             x86_64                                             2.5-33.el7                                                     base                                                         457 k
 python-IPy                                                         noarch                                             0.75-6.el7                                                     base                                                          32 k
 setools-libs                                                       x86_64                                             3.3.8-4.el7                                                    base                                                         620 k
Updating for dependencies:
 audit                                                              x86_64                                             2.8.5-4.el7                                                    base                                                         256 k
 audit-libs                                                         x86_64                                             2.8.5-4.el7                                                    base                                                         102 k
 policycoreutils                                                    x86_64                                             2.5-33.el7                                                     base                                                         916 k

Transaction Summary
=========================================================================================================================================================================================================================================================
Install  1 Package  (+10 Dependent packages)
Upgrade             (  3 Dependent packages)

Total download size: 90 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/14): audit-libs-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                         | 102 kB  00:00:00     
(2/14): audit-libs-python-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                  |  76 kB  00:00:00     
(3/14): audit-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                              | 256 kB  00:00:00     
(4/14): checkpolicy-2.5-8.el7.x86_64.rpm                                                                                                                                                                                          | 295 kB  00:00:00     
(5/14): container-selinux-2.107-3.el7.noarch.rpm                                                                                                                                                                                  |  39 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/containerd.io-1.2.10-3.2.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY                                                             ] 4.7 MB/s |  22 MB  00:00:14 ETA 
Public key for containerd.io-1.2.10-3.2.el7.x86_64.rpm is not installed
(6/14): containerd.io-1.2.10-3.2.el7.x86_64.rpm                                                                                                                                                                                   |  23 MB  00:00:03     
(7/14): libcgroup-0.41-21.el7.x86_64.rpm                                                                                                                                                                                          |  66 kB  00:00:00     
(8/14): libsemanage-python-2.5-14.el7.x86_64.rpm                                                                                                                                                                                  | 113 kB  00:00:00     
(9/14): python-IPy-0.75-6.el7.noarch.rpm                                                                                                                                                                                          |  32 kB  00:00:00     
(10/14): docker-ce-cli-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                   |  39 MB  00:00:04     
(11/14): setools-libs-3.3.8-4.el7.x86_64.rpm                                                                                                                                                                                      | 620 kB  00:00:03     
(12/14): policycoreutils-2.5-33.el7.x86_64.rpm                                                                                                                                                                                    | 916 kB  00:00:06     
(13/14): policycoreutils-python-2.5-33.el7.x86_64.rpm                                                                                                                                                                             | 457 kB  00:00:08     
(14/14): docker-ce-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                       |  24 MB  00:00:20     
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                    4.3 MB/s |  90 MB  00:00:20     
Retrieving key from https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
 From       : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Updating   : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                    2/17 
  Installing : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                         3/17 
  Installing : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 4/17 
  Installing : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                      5/17 
  Installing : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                 6/17 
  Installing : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         7/17 
  Installing : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Installing : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 9/17 
  Installing : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                            10/17 
  Installing : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                              11/17 
setsebool:  SELinux is disabled.
  Installing : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 12/17 
  Installing : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                    13/17 
  Updating   : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                            14/17 
  Cleanup    : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Cleanup    : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Cleanup    : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 
  Verifying  : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Verifying  : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                             2/17 
  Verifying  : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                             3/17 
  Verifying  : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                     4/17 
  Verifying  : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 5/17 
  Verifying  : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 6/17 
  Verifying  : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                               7/17 
  Verifying  : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Verifying  : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         9/17 
  Verifying  : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                   10/17 
  Verifying  : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 11/17 
  Verifying  : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                12/17 
  Verifying  : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                     13/17 
  Verifying  : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                        14/17 
  Verifying  : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Verifying  : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Verifying  : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 

Installed:
  docker-ce.x86_64 3:19.03.5-3.el7                                                                                                                                                                                                                       

Dependency Installed:
  audit-libs-python.x86_64 0:2.8.5-4.el7    checkpolicy.x86_64 0:2.5-8.el7                container-selinux.noarch 2:2.107-3.el7    containerd.io.x86_64 0:1.2.10-3.2.el7    docker-ce-cli.x86_64 1:19.03.5-3.el7    libcgroup.x86_64 0:0.41-21.el7   
  libsemanage-python.x86_64 0:2.5-14.el7    policycoreutils-python.x86_64 0:2.5-33.el7    python-IPy.noarch 0:0.75-6.el7            setools-libs.x86_64 0:3.3.8-4.el7       

Dependency Updated:
  audit.x86_64 0:2.8.5-4.el7                                                    audit-libs.x86_64 0:2.8.5-4.el7                                                    policycoreutils.x86_64 0:2.5-33.el7                                                   

Complete!
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# yum -y install docker-ce

[root@node202.yinzhengjie.org.cn ~]# yum -y install docker-ce
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                  | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/x86_64/updateinfo                                                                                                                                                                                         |   55 B  00:00:00     
(2/2): docker-ce-stable/x86_64/primary_db                                                                                                                                                                                         |  37 kB  00:00:00     
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.5-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: libcgroup for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.107-3.el7 will be installed
--> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.107-3.el7.noarch
---> Package containerd.io.x86_64 0:1.2.10-3.2.el7 will be installed
---> Package docker-ce-cli.x86_64 1:19.03.5-3.el7 will be installed
---> Package libcgroup.x86_64 0:0.41-21.el7 will be installed
--> Running transaction check
---> Package policycoreutils-python.x86_64 0:2.5-33.el7 will be installed
--> Processing Dependency: policycoreutils = 2.5-33.el7 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Running transaction check
---> Package audit-libs-python.x86_64 0:2.8.5-4.el7 will be installed
--> Processing Dependency: audit-libs(x86-64) = 2.8.5-4.el7 for package: audit-libs-python-2.8.5-4.el7.x86_64
---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed
---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed
---> Package policycoreutils.x86_64 0:2.5-29.el7 will be updated
---> Package policycoreutils.x86_64 0:2.5-33.el7 will be an update
---> Package python-IPy.noarch 0:0.75-6.el7 will be installed
---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed
--> Running transaction check
---> Package audit-libs.x86_64 0:2.8.4-4.el7 will be updated
--> Processing Dependency: audit-libs(x86-64) = 2.8.4-4.el7 for package: audit-2.8.4-4.el7.x86_64
---> Package audit-libs.x86_64 0:2.8.5-4.el7 will be an update
--> Running transaction check
---> Package audit.x86_64 0:2.8.4-4.el7 will be updated
---> Package audit.x86_64 0:2.8.5-4.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================================
 Package                                                            Arch                                               Version                                                        Repository                                                    Size
=========================================================================================================================================================================================================================================================
Installing:
 docker-ce                                                          x86_64                                             3:19.03.5-3.el7                                                docker-ce-stable                                              24 M
Installing for dependencies:
 audit-libs-python                                                  x86_64                                             2.8.5-4.el7                                                    base                                                          76 k
 checkpolicy                                                        x86_64                                             2.5-8.el7                                                      base                                                         295 k
 container-selinux                                                  noarch                                             2:2.107-3.el7                                                  extras                                                        39 k
 containerd.io                                                      x86_64                                             1.2.10-3.2.el7                                                 docker-ce-stable                                              23 M
 docker-ce-cli                                                      x86_64                                             1:19.03.5-3.el7                                                docker-ce-stable                                              39 M
 libcgroup                                                          x86_64                                             0.41-21.el7                                                    base                                                          66 k
 libsemanage-python                                                 x86_64                                             2.5-14.el7                                                     base                                                         113 k
 policycoreutils-python                                             x86_64                                             2.5-33.el7                                                     base                                                         457 k
 python-IPy                                                         noarch                                             0.75-6.el7                                                     base                                                          32 k
 setools-libs                                                       x86_64                                             3.3.8-4.el7                                                    base                                                         620 k
Updating for dependencies:
 audit                                                              x86_64                                             2.8.5-4.el7                                                    base                                                         256 k
 audit-libs                                                         x86_64                                             2.8.5-4.el7                                                    base                                                         102 k
 policycoreutils                                                    x86_64                                             2.5-33.el7                                                     base                                                         916 k

Transaction Summary
=========================================================================================================================================================================================================================================================
Install  1 Package  (+10 Dependent packages)
Upgrade             (  3 Dependent packages)

Total download size: 90 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/14): audit-libs-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                         | 102 kB  00:00:01     
(2/14): audit-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                              | 256 kB  00:00:01     
(3/14): container-selinux-2.107-3.el7.noarch.rpm                                                                                                                                                                                  |  39 kB  00:00:00     
(4/14): audit-libs-python-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                  |  76 kB  00:00:01     
(5/14): checkpolicy-2.5-8.el7.x86_64.rpm                                                                                                                                                                                          | 295 kB  00:00:03     
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/containerd.io-1.2.10-3.2.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY                                                             ] 3.8 MB/s |  34 MB  00:00:14 ETA 
Public key for containerd.io-1.2.10-3.2.el7.x86_64.rpm is not installed
(6/14): containerd.io-1.2.10-3.2.el7.x86_64.rpm                                                                                                                                                                                   |  23 MB  00:00:15     
(7/14): libcgroup-0.41-21.el7.x86_64.rpm                                                                                                                                                                                          |  66 kB  00:00:00     
(8/14): libsemanage-python-2.5-14.el7.x86_64.rpm                                                                                                                                                                                  | 113 kB  00:00:00     
(9/14): python-IPy-0.75-6.el7.noarch.rpm                                                                                                                                                                                          |  32 kB  00:00:00     
(10/14): policycoreutils-python-2.5-33.el7.x86_64.rpm                                                                                                                                                                             | 457 kB  00:00:00     
(11/14): setools-libs-3.3.8-4.el7.x86_64.rpm                                                                                                                                                                                      | 620 kB  00:00:00     
(12/14): policycoreutils-2.5-33.el7.x86_64.rpm                                                                                                                                                                                    | 916 kB  00:00:02     
(13/14): docker-ce-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                       |  24 MB  00:00:21     
(14/14): docker-ce-cli-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                   |  39 MB  00:00:07     
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                    3.7 MB/s |  90 MB  00:00:24     
Retrieving key from https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
 From       : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Updating   : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                    2/17 
  Installing : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                         3/17 
  Installing : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 4/17 
  Installing : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                      5/17 
  Installing : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                 6/17 
  Installing : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         7/17 
  Installing : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Installing : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 9/17 
  Installing : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                            10/17 
  Installing : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                              11/17 
setsebool:  SELinux is disabled.
  Installing : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 12/17 
  Installing : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                    13/17 
  Updating   : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                            14/17 
  Cleanup    : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Cleanup    : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Cleanup    : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 
  Verifying  : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Verifying  : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                             2/17 
  Verifying  : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                             3/17 
  Verifying  : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                     4/17 
  Verifying  : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 5/17 
  Verifying  : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 6/17 
  Verifying  : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                               7/17 
  Verifying  : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Verifying  : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         9/17 
  Verifying  : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                   10/17 
  Verifying  : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 11/17 
  Verifying  : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                12/17 
  Verifying  : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                     13/17 
  Verifying  : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                        14/17 
  Verifying  : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Verifying  : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Verifying  : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 

Installed:
  docker-ce.x86_64 3:19.03.5-3.el7                                                                                                                                                                                                                       

Dependency Installed:
  audit-libs-python.x86_64 0:2.8.5-4.el7    checkpolicy.x86_64 0:2.5-8.el7                container-selinux.noarch 2:2.107-3.el7    containerd.io.x86_64 0:1.2.10-3.2.el7    docker-ce-cli.x86_64 1:19.03.5-3.el7    libcgroup.x86_64 0:0.41-21.el7   
  libsemanage-python.x86_64 0:2.5-14.el7    policycoreutils-python.x86_64 0:2.5-33.el7    python-IPy.noarch 0:0.75-6.el7            setools-libs.x86_64 0:3.3.8-4.el7       

Dependency Updated:
  audit.x86_64 0:2.8.5-4.el7                                                    audit-libs.x86_64 0:2.8.5-4.el7                                                    policycoreutils.x86_64 0:2.5-33.el7                                                   

Complete!
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# yum -y install docker-ce

[root@node203.yinzhengjie.org.cn ~]# yum -y install docker-ce
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                  | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/x86_64/updateinfo                                                                                                                                                                                         |   55 B  00:00:00     
(2/2): docker-ce-stable/x86_64/primary_db                                                                                                                                                                                         |  37 kB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.5-3.el7 will be installed
--> Processing Dependency: container-selinux >= 2:2.74 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: containerd.io >= 1.2.2-3 for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: docker-ce-cli for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Processing Dependency: libcgroup for package: 3:docker-ce-19.03.5-3.el7.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.107-3.el7 will be installed
--> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.107-3.el7.noarch
---> Package containerd.io.x86_64 0:1.2.10-3.2.el7 will be installed
---> Package docker-ce-cli.x86_64 1:19.03.5-3.el7 will be installed
---> Package libcgroup.x86_64 0:0.41-21.el7 will be installed
--> Running transaction check
---> Package policycoreutils-python.x86_64 0:2.5-33.el7 will be installed
--> Processing Dependency: policycoreutils = 2.5-33.el7 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-33.el7.x86_64
--> Running transaction check
---> Package audit-libs-python.x86_64 0:2.8.5-4.el7 will be installed
--> Processing Dependency: audit-libs(x86-64) = 2.8.5-4.el7 for package: audit-libs-python-2.8.5-4.el7.x86_64
---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed
---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed
---> Package policycoreutils.x86_64 0:2.5-29.el7 will be updated
---> Package policycoreutils.x86_64 0:2.5-33.el7 will be an update
---> Package python-IPy.noarch 0:0.75-6.el7 will be installed
---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed
--> Running transaction check
---> Package audit-libs.x86_64 0:2.8.4-4.el7 will be updated
--> Processing Dependency: audit-libs(x86-64) = 2.8.4-4.el7 for package: audit-2.8.4-4.el7.x86_64
---> Package audit-libs.x86_64 0:2.8.5-4.el7 will be an update
--> Running transaction check
---> Package audit.x86_64 0:2.8.4-4.el7 will be updated
---> Package audit.x86_64 0:2.8.5-4.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=========================================================================================================================================================================================================================================================
 Package                                                            Arch                                               Version                                                        Repository                                                    Size
=========================================================================================================================================================================================================================================================
Installing:
 docker-ce                                                          x86_64                                             3:19.03.5-3.el7                                                docker-ce-stable                                              24 M
Installing for dependencies:
 audit-libs-python                                                  x86_64                                             2.8.5-4.el7                                                    base                                                          76 k
 checkpolicy                                                        x86_64                                             2.5-8.el7                                                      base                                                         295 k
 container-selinux                                                  noarch                                             2:2.107-3.el7                                                  extras                                                        39 k
 containerd.io                                                      x86_64                                             1.2.10-3.2.el7                                                 docker-ce-stable                                              23 M
 docker-ce-cli                                                      x86_64                                             1:19.03.5-3.el7                                                docker-ce-stable                                              39 M
 libcgroup                                                          x86_64                                             0.41-21.el7                                                    base                                                          66 k
 libsemanage-python                                                 x86_64                                             2.5-14.el7                                                     base                                                         113 k
 policycoreutils-python                                             x86_64                                             2.5-33.el7                                                     base                                                         457 k
 python-IPy                                                         noarch                                             0.75-6.el7                                                     base                                                          32 k
 setools-libs                                                       x86_64                                             3.3.8-4.el7                                                    base                                                         620 k
Updating for dependencies:
 audit                                                              x86_64                                             2.8.5-4.el7                                                    base                                                         256 k
 audit-libs                                                         x86_64                                             2.8.5-4.el7                                                    base                                                         102 k
 policycoreutils                                                    x86_64                                             2.5-33.el7                                                     base                                                         916 k

Transaction Summary
=========================================================================================================================================================================================================================================================
Install  1 Package  (+10 Dependent packages)
Upgrade             (  3 Dependent packages)

Total download size: 90 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/14): audit-libs-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                         | 102 kB  00:00:00     
(2/14): audit-libs-python-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                  |  76 kB  00:00:00     
(3/14): container-selinux-2.107-3.el7.noarch.rpm                                                                                                                                                                                  |  39 kB  00:00:00     
(4/14): audit-2.8.5-4.el7.x86_64.rpm                                                                                                                                                                                              | 256 kB  00:00:01     
(5/14): checkpolicy-2.5-8.el7.x86_64.rpm                                                                                                                                                                                          | 295 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/docker-ce-19.03.5-3.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY                                                                  ] 3.8 MB/s |  24 MB  00:00:17 ETA 
Public key for docker-ce-19.03.5-3.el7.x86_64.rpm is not installed
(6/14): docker-ce-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                        |  24 MB  00:00:03     
(7/14): libcgroup-0.41-21.el7.x86_64.rpm                                                                                                                                                                                          |  66 kB  00:00:00     
(8/14): libsemanage-python-2.5-14.el7.x86_64.rpm                                                                                                                                                                                  | 113 kB  00:00:00     
(9/14): python-IPy-0.75-6.el7.noarch.rpm                                                                                                                                                                                          |  32 kB  00:00:00     
(10/14): policycoreutils-python-2.5-33.el7.x86_64.rpm                                                                                                                                                                             | 457 kB  00:00:01     
(11/14): policycoreutils-2.5-33.el7.x86_64.rpm                                                                                                                                                                                    | 916 kB  00:00:01     
(12/14): setools-libs-3.3.8-4.el7.x86_64.rpm                                                                                                                                                                                      | 620 kB  00:00:01     
(13/14): docker-ce-cli-19.03.5-3.el7.x86_64.rpm                                                                                                                                                                                   |  39 MB  00:00:08     
(14/14): containerd.io-1.2.10-3.2.el7.x86_64.rpm                                                                                                                                                                                  |  23 MB  00:00:19     
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                    4.5 MB/s |  90 MB  00:00:20     
Retrieving key from https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Importing GPG key 0x621E9F35:
 Userid     : "Docker Release (CE rpm) <docker@docker.com>"
 Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
 From       : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Updating   : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                    2/17 
  Installing : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                         3/17 
  Installing : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 4/17 
  Installing : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                      5/17 
  Installing : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                 6/17 
  Installing : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         7/17 
  Installing : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Installing : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 9/17 
  Installing : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                            10/17 
  Installing : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                              11/17 
setsebool:  SELinux is disabled.
  Installing : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 12/17 
  Installing : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                    13/17 
  Updating   : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                            14/17 
  Cleanup    : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Cleanup    : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Cleanup    : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 
  Verifying  : audit-libs-2.8.5-4.el7.x86_64                                                                                                                                                                                                        1/17 
  Verifying  : policycoreutils-python-2.5-33.el7.x86_64                                                                                                                                                                                             2/17 
  Verifying  : audit-2.8.5-4.el7.x86_64                                                                                                                                                                                                             3/17 
  Verifying  : 3:docker-ce-19.03.5-3.el7.x86_64                                                                                                                                                                                                     4/17 
  Verifying  : audit-libs-python-2.8.5-4.el7.x86_64                                                                                                                                                                                                 5/17 
  Verifying  : libsemanage-python-2.5-14.el7.x86_64                                                                                                                                                                                                 6/17 
  Verifying  : 2:container-selinux-2.107-3.el7.noarch                                                                                                                                                                                               7/17 
  Verifying  : python-IPy-0.75-6.el7.noarch                                                                                                                                                                                                         8/17 
  Verifying  : checkpolicy-2.5-8.el7.x86_64                                                                                                                                                                                                         9/17 
  Verifying  : policycoreutils-2.5-33.el7.x86_64                                                                                                                                                                                                   10/17 
  Verifying  : containerd.io-1.2.10-3.2.el7.x86_64                                                                                                                                                                                                 11/17 
  Verifying  : 1:docker-ce-cli-19.03.5-3.el7.x86_64                                                                                                                                                                                                12/17 
  Verifying  : setools-libs-3.3.8-4.el7.x86_64                                                                                                                                                                                                     13/17 
  Verifying  : libcgroup-0.41-21.el7.x86_64                                                                                                                                                                                                        14/17 
  Verifying  : policycoreutils-2.5-29.el7.x86_64                                                                                                                                                                                                   15/17 
  Verifying  : audit-2.8.4-4.el7.x86_64                                                                                                                                                                                                            16/17 
  Verifying  : audit-libs-2.8.4-4.el7.x86_64                                                                                                                                                                                                       17/17 

Installed:
  docker-ce.x86_64 3:19.03.5-3.el7                                                                                                                                                                                                                       

Dependency Installed:
  audit-libs-python.x86_64 0:2.8.5-4.el7    checkpolicy.x86_64 0:2.5-8.el7                container-selinux.noarch 2:2.107-3.el7    containerd.io.x86_64 0:1.2.10-3.2.el7    docker-ce-cli.x86_64 1:19.03.5-3.el7    libcgroup.x86_64 0:0.41-21.el7   
  libsemanage-python.x86_64 0:2.5-14.el7    policycoreutils-python.x86_64 0:2.5-33.el7    python-IPy.noarch 0:0.75-6.el7            setools-libs.x86_64 0:3.3.8-4.el7       

Dependency Updated:
  audit.x86_64 0:2.8.5-4.el7                                                    audit-libs.x86_64 0:2.8.5-4.el7                                                    policycoreutils.x86_64 0:2.5-33.el7                                                   

Complete!
[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# yum -y install docker-ce

3>.修改docker的默认策略,重新将FORWARD链的默认策略设置为ACCEPT

  docker自1.13版起会自动设置iptable的FORWARD默认策略为DROP,这可能会影响kubernetes集群依赖的报文转发功能,因此,需要在docker服务启动后,重新将FORWARD链的默认策略设置为ACCEPT。

  如下图所示,修改"/usr/lib/systemd/system/docker.service"文件,在"ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock"一行之后新增一行"ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT"(意思是docker服务器启动成功后会执行执行该命令)

4>.为docker设置代理(一旦我们为docker服务器设置代理后再去配置阿里云加速就没有多大意义了,因此我这里并没有配置镜像加速,因为默认会走代理服务器)

  若要通过默认的k8s.gcr.io镜像仓库(实际上国内的网络是无法直接访问Google的资源)获取kubernetes系统组件的相关镜像,需要配置docker Unit File(/usr/lib/systemd/system/docker.service)中的Environment变量,为其定义合理的HTTPS_PROXY(说白了就是你自己的VPN服务器,如果没有VPN的话还是乖乖的使用阿里云镜像吧),当然我们访问跟本地网络或者node的IP时则无需代理,使用NO_PROXY指定不代理的网段,格式如下(具体配置参考下图):
    Environment="HTTPS_PROXY=PROTOCOL://HOST:PORT"
    Environment="NO_PROXY=127.0.0.0/8,172.200.0.0/21"

  温馨提示:
    如果你不想通过k8s.gcr.io镜像仓库下载那就可以直接忽略此步骤,因为阿里云已经帮咱们提前下载好k8s镜像啦,我们直接去阿里云下载即可,使用阿里云下载需要配置阿里云加速,具体步骤可参考:"https://www.cnblogs.com/yinzhengjie/p/12182645.html"

5>.启动docker

[root@master200.yinzhengjie.org.cn ~]# systemctl daemon-reload 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl start docker.service 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl enable docker.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep docker
docker.service                                enabled 
docker.socket                                 disabled
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl status docker.service 
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-02-04 11:30:27 CST; 1min 4s ago
     Docs: https://docs.docker.com
 Main PID: 6182 (dockerd)
   CGroup: /system.slice/docker.service
           └─6182 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.706191791+08:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.706203607+08:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.706210087+08:00" level=info msg="ClientConn switching balancer \to "pick_first\"" module=grpc
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.732793779+08:00" level=info msg="Loading containers: start."
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.819420862+08:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a...rred IP address"
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.858229542+08:00" level=info msg="Loading containers: done."
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.871357290+08:00" level=info msg="Docker daemon" commit=633a0ea graphdriver(s)=overlay2 version=19.03.5
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.871433828+08:00" level=info msg="Daemon has completed initialization"
Feb 04 11:30:27 master200.yinzhengjie.org.cn dockerd[6182]: time="2020-02-04T11:30:27.893618790+08:00" level=info msg="API listen on /var/run/docker.sock"
Feb 04 11:30:27 master200.yinzhengjie.org.cn systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
[root@master200.yinzhengjie.org.cn ~]#

6>.查看docker的代理信息

[root@master200.yinzhengjie.org.cn ~]# docker info 
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-957.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.84GiB
 Name: master200.yinzhengjie.org.cn
 ID: CSHT:CVB3:JFYB:GQ77:FYDH:X3UJ:B2SH:5WEX:5QNE:NABW:MJZM:K2T2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTPS Proxy: http://www.yinzhengjie.org.cn:10086
 No Proxy: 127.0.0.0/8,172.200.0.0/21
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# docker info

7>.查看防火墙FORWARD链的默认策略

[root@master200.yinzhengjie.org.cn ~]# iptables -vnL
Chain INPUT (policy ACCEPT 247 packets, 17304 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 147 packets, 18292 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# iptables -vnL

8**>.将master200的docker文件拷贝到其它节点启动docker并设置为开机自启动**

[root@master200.yinzhengjie.org.cn ~]# scp /usr/lib/systemd/system/docker.service  node201.yinzhengjie.org.cn:/usr/lib/systemd/system/docker.service
root@node201.yinzhengjie.org.cn's password: 
docker.service                                                                                                                                                                                                          100% 1846     1.2MB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /usr/lib/systemd/system/docker.service  node202.yinzhengjie.org.cn:/usr/lib/systemd/system/docker.service
root@node202.yinzhengjie.org.cn's password: 
docker.service                                                                                                                                                                                                          100% 1846     1.5MB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /usr/lib/systemd/system/docker.service  node203.yinzhengjie.org.cn:/usr/lib/systemd/system/docker.service
root@node203.yinzhengjie.org.cn's password: 
docker.service                                                                                                                                                                                                          100% 1846   790.6KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]#

9>.启动docker后验证的网络相关的内核参数(了解一下即可,如果你遇到了类似问题安装下面的方法解决即可)

[root@master200.yinzhengjie.org.cn ~]# sysctl -a | grep bridge
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-filter-pppoe-tagged = 0
net.bridge.bridge-nf-filter-vlan-tagged = 0
net.bridge.bridge-nf-pass-vlan-input-dev = 0
sysctl: reading key "net.ipv6.conf.all.stable_secret"
sysctl: reading key "net.ipv6.conf.bond0.stable_secret"
sysctl: reading key "net.ipv6.conf.bond1.stable_secret"
sysctl: reading key "net.ipv6.conf.default.stable_secret"
sysctl: reading key "net.ipv6.conf.docker0.stable_secret"
sysctl: reading key "net.ipv6.conf.eth0.stable_secret"
sysctl: reading key "net.ipv6.conf.eth1.stable_secret"
sysctl: reading key "net.ipv6.conf.eth2.stable_secret"
sysctl: reading key "net.ipv6.conf.eth3.stable_secret"
sysctl: reading key "net.ipv6.conf.lo.stable_secret"
[root@master200.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# sysctl -p /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@node203.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# sysctl -a | grep bridge

三.安装kubernetes相关的软件包并初始化集群

1>.编写k8s的软件源仓库的配置文件

[root@master200.yinzhengjie.org.cn ~]# vim /etc/yum.repos.d/kubernetes.repo
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Yinzhengjie's Kubernetes Repository
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
kubernetes                                                                                                                                                                                                                         | 1.4 kB  00:00:00     
kubernetes/primary                                                                                                                                                                                                                 |  63 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                        460/460
repo id                                                                                                             repo name                                                                                                                       status
base/7/x86_64                                                                                                       CentOS-7 - Base                                                                                                                 10,097
docker-ce-stable/x86_64                                                                                             Docker CE Stable - x86_64                                                                                                           63
extras/7/x86_64                                                                                                     CentOS-7 - Extras                                                                                                                  323
kubernetes                                                                                                          Yinzhengjie's Kubernetes Repository                                                                                                460
updates/7/x86_64                                                                                                    CentOS-7 - Updates                                                                                                               1,117
repolist: 12,060
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum repolist

[root@master200.yinzhengjie.org.cn ~]# yum list all | grep "^kube"
kubeadm.x86_64                              1.17.2-0                   kubernetes
kubectl.x86_64                              1.17.2-0                   kubernetes
kubelet.x86_64                              1.17.2-0                   kubernetes
kubernetes.x86_64                           1.5.2-0.7.git269f928.el7   extras   
kubernetes-client.x86_64                    1.5.2-0.7.git269f928.el7   extras   
kubernetes-cni.x86_64                       0.7.5-0                    kubernetes
kubernetes-master.x86_64                    1.5.2-0.7.git269f928.el7   extras   
kubernetes-node.x86_64                      1.5.2-0.7.git269f928.el7   extras   
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum list all | grep "^kube"

2>.安装软件包并查看安装的版本

[root@master200.yinzhengjie.org.cn ~]# yum -y install kubeadm kubectl kubelet
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.tuna.tsinghua.edu.cn
 * updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
---> Package kubectl.x86_64 0:1.17.2-0 will be installed
---> Package kubelet.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.17.2-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.17.2-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================================================================================================
 Package                                                              Arch                                                 Version                                                         Repository                                                Size
==========================================================================================================================================================================================================================================================
Installing:
 kubeadm                                                              x86_64                                               1.17.2-0                                                        kubernetes                                               8.7 M
 kubectl                                                              x86_64                                               1.17.2-0                                                        kubernetes                                               9.4 M
 kubelet                                                              x86_64                                               1.17.2-0                                                        kubernetes                                                20 M
Installing for dependencies:
 conntrack-tools                                                      x86_64                                               1.4.4-5.el7_7.2                                                 updates                                                  187 k
 cri-tools                                                            x86_64                                               1.13.0-0                                                        kubernetes                                               5.1 M
 kubernetes-cni                                                       x86_64                                               0.7.5-0                                                         kubernetes                                                10 M
 libnetfilter_cthelper                                                x86_64                                               1.0.0-10.el7_7.1                                                updates                                                   18 k
 libnetfilter_cttimeout                                               x86_64                                               1.0.0-6.el7_7.1                                                 updates                                                   18 k
 libnetfilter_queue                                                   x86_64                                               1.0.2-2.el7_2                                                   base                                                      23 k
 socat                                                                x86_64                                               1.7.3.2-2.el7                                                   base                                                     290 k

Transaction Summary
==========================================================================================================================================================================================================================================================
Install  3 Packages (+7 Dependent packages)

Total download size: 54 M
Installed size: 243 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-5.el7_7.2.x86_64.rpm                                                                                                                                                                                 | 187 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY             ] 2.0 MB/s | 1.7 MB  00:00:26 ETA 
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                                                                                                             | 5.1 MB  00:00:00     
(3/10): b44630896c69cd411db53be1d5cb5ae899a40aba7c0766317ea904390fcfc45b-kubectl-1.17.2-0.x86_64.rpm                                                                                                                               | 9.4 MB  00:00:01     
(4/10): 105d89f0607c7baf91305ba352e78000bd20aad5cdf706bffff3b31cd546dbf3-kubeadm-1.17.2-0.x86_64.rpm                                                                                                                               | 8.7 MB  00:00:02     
(5/10): libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64.rpm                                                                                                                                                                          |  18 kB  00:00:00     
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                                                                                                                |  23 kB  00:00:00     
(7/10): libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64.rpm                                                                                                                                                                          |  18 kB  00:00:00     
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                                                                                                             | 290 kB  00:00:01     
(9/10): 3ee7f2dff78e6fbb3ac3af8acb1a907f4bec1b1ef4cf627cbe02fa553707f2e9-kubelet-1.17.2-0.x86_64.rpm                                                                                                                               |  20 MB  00:00:02     
(10/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                                                                                                        |  10 MB  00:00:03     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                     9.1 MB/s |  54 MB  00:00:05     
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                         1/10 
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                            2/10 
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                             3/10 
  Installing : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                               4/10 
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                               5/10 
  Installing : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                         6/10 
  Installing : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                7/10 
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                         8/10 
  Installing : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                               9/10 
  Installing : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                              10/10 
  Verifying  : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                         1/10 
  Verifying  : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                2/10 
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                               3/10 
  Verifying  : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                               4/10 
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                             5/10 
  Verifying  : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                               6/10 
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                         7/10 
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                            8/10 
  Verifying  : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                               9/10 
  Verifying  : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                        10/10 

Installed:
  kubeadm.x86_64 0:1.17.2-0                                                          kubectl.x86_64 0:1.17.2-0                                                          kubelet.x86_64 0:1.17.2-0                                                         

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-5.el7_7.2  cri-tools.x86_64 0:1.13.0-0  kubernetes-cni.x86_64 0:0.7.5-0  libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1  libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1  libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 
  socat.x86_64 0:1.7.3.2-2.el7             

Complete!
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# yum -y install kubeadm kubectl kubelet

[root@master200.yinzhengjie.org.cn ~]# rpm -ql kubelet
/etc/kubernetes/manifests
/etc/sysconfig/kubelet
/usr/bin/kubelet
/usr/lib/systemd/system/kubelet.service
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# rpm -ql kubeadm
/usr/bin/kubeadm
/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# rpm -ql kubectl
/usr/bin/kubectl
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# rpm -q kubeadm 
kubeadm-1.17.2-0.x86_64
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# rpm -q kubectl
kubectl-1.17.2-0.x86_64
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# rpm -q kubelet
kubelet-1.17.2-0.x86_64
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               disabled
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl status kubelet 
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: inactive (dead)
     Docs: https://kubernetes.io/docs/
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl enable kubelet 
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               enabled 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl start kubelet 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# systemctl status kubelet 
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: active (running) since Wed 2020-02-05 05:12:34 CST; 2s ago
     Docs: https://kubernetes.io/docs/
 Main PID: 5675 (kubelet)
    Tasks: 11
   Memory: 117.3M
   CGroup: /system.slice/kubelet.service
           └─5675 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-im...

Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: I0205 05:12:35.006504    5675 remote_image.go:50] parsed scheme: ""
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: I0205 05:12:35.006511    5675 remote_image.go:50] scheme "" not registered, fallback to default scheme
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: I0205 05:12:35.006519    5675 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0  <nil>}] <nil>}
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: I0205 05:12:35.006523    5675 clientconn.go:577] ClientConn switching balancer to "pick_first"
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:35.933510    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.200.1.200:6443/api/v1/nodes?fieldSelector=metadata.n...
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:35.934154    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.200.1.200:6443/api/v1/services?li...nnection refused
Feb 05 05:12:35 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:35.935345    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.200.1.200:6443/api/v1/pods?fieldSelector=spec...
Feb 05 05:12:36 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:36.934349    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.200.1.200:6443/api/v1/nodes?fieldSelector=metadata.n...
Feb 05 05:12:36 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:36.935197    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.200.1.200:6443/api/v1/services?li...nnection refused
Feb 05 05:12:36 master200.yinzhengjie.org.cn kubelet[5675]: E0205 05:12:36.937286    5675 reflector.go:153] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.200.1.200:6443/api/v1/pods?fieldSelector=spec...
Hint: Some lines were ellipsized, use -l to show in full.
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# systemctl enable kubelet

3>.设置忽略swap启用的状态错误(如果你未禁用swap设备则需要编辑kubelet的配置文件,我们在安装操作系统时就压根没有分配swap分区,因此该步骤直接忽略即可)

[root@master200.yinzhengjie.org.cn ~]# vim /etc/sysconfig/kubelet 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS="--fail-swap-on=false"        #该参数表示尽管swap分区是启用的也不报错,如果不设置默认情况下如果你的服务器启用了swap分区那么初始化时会报错的
[root@master200.yinzhengjie.org.cn ~]#

4>.先将镜像文件下载到本地

[root@master200.yinzhengjie.org.cn ~]# kubeadm config print init-defaults
W0204 12:49:25.072012   10194 validation.go:28] Cannot validate kubelet config - no validator is available
W0204 12:49:25.072058   10194 validation.go:28] Cannot validate kube-proxy config - no validator is available
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 1.2.3.4
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: master200.yinzhengjie.org.cn
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.17.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# kubeadm config print init-defaults          #查看kubeadm的默认配置

  如果你有对docker配置了代理(即指定了VPN服务器)那么就可以使用Google官方提供的镜像仓库进行下载,执行如下命令即可:
    [root@master200.yinzhengjie.org.cn ~]# kubeadm config images pull

  如上图所示,默认情况下是无法直接访问Google的镜像仓库"k8s.gcr.io",此时咱们有三种解决方案:
    (1)在互联网上找一些免费的VPN进行FQ操作或者在网上购买收费的VPN,推荐使用收费版本的,因为免费版本的VPN并不太稳定而且下载速度也并不是很理想,付费的VPN相对来说稳定性比较强,但是这种情况存在一个风险,因为你的所有操作都会被提供VPN的服务商在后台记录;
    (2)在互联网上购买VPS服务器,自己搭建VPN,前提是你购买的VPS可以访问国外的各种网站,直接购买国内的阿里云服务器可以你搭建出来VPN也是白搭,这种方案也需要掏钱,但相比于上面的那种方式安全性较强,因为你访问的记录在你的VPS中有记录,你想干啥就干啥;
    (3)我们的目的就是下载Google的镜像而已,可以在其它国内站点下载即可,而且很多国人已经很热心的提供了下载连接,如果你没有经常FQ的需求其实没有必要采取上面两种方案(而且有很多公司也会为员工购买FQ的工具,自己搭建VPN服务器的一般就是一些运维或开发人员喜欢干这事)。

  本篇博客采取第三种方案,直接在国内的其它网站(比如阿里云的镜像网站)下载需要的镜像,下载后给镜像重新更名即可,具体操作请参考我笔记:
    K8S镜像下载报错解决方案(使用阿里云镜像去下载kubeadm需要的镜像文件):https://www.cnblogs.com/yinzhengjie/p/12258215.html
  镜像成功下载到本地后,可以直接使用docker命令查看本地镜像,如下图所示。

5>.初始化集群(初始化过程会自动去下载镜像,但上一步骤我们已经把镜像提前下载到本地啦,因此不会去官网下载镜像啦,这一步骤也会快很多)

[root@master200.yinzhengjie.org.cn ~]# kubeadm init --kubernetes-version="v1.17.2" --pod-network-cidr="10.244.0.0/16" --dry-run
W0204 19:37:26.784651   31963 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0204 19:37:26.784688   31963 validation.go:28] Cannot validate kubelet config - no validator is available
[init] Using Kubernetes version: v1.17.2
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Would pull the required images (like 'kubeadm config images pull')
[kubelet-start] Writing kubelet environment file with flags to file "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826/config.yaml"
[certs] Using certificateDir folder "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [master200.yinzhengjie.org.cn kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 172.200.1.200]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [master200.yinzhengjie.org.cn localhost] and IPs [172.200.1.200 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [master200.yinzhengjie.org.cn localhost] and IPs [172.200.1.200 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0204 19:37:28.996203   31963 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0204 19:37:28.997161   31963 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[dryrun] Would ensure that "/var/lib/etcd" directory is present
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826"
[dryrun] Wrote certificates, kubeconfig files and control plane manifests to the "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826" directory
[dryrun] The certificates or kubeconfig files would not be printed due to their sensitive nature
[dryrun] Please examine the "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826" directory for details about what would be written
[dryrun] Would write file "/etc/kubernetes/manifests/kube-apiserver.yaml" with content:
    apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: null
      labels:
        component: kube-apiserver
        tier: control-plane
      name: kube-apiserver
      namespace: kube-system
    spec:
      containers:
      - command:
        - kube-apiserver
        - --advertise-address=172.200.1.200
        - --allow-privileged=true
        - --authorization-mode=Node,RBAC
        - --client-ca-file=/etc/kubernetes/pki/ca.crt
        - --enable-admission-plugins=NodeRestriction
        - --enable-bootstrap-token-auth=true
        - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
        - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
        - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
        - --etcd-servers=https://127.0.0.1:2379
        - --insecure-port=0
        - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
        - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
        - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
        - --requestheader-allowed-names=front-proxy-client
        - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
        - --requestheader-extra-headers-prefix=X-Remote-Extra-
        - --requestheader-group-headers=X-Remote-Group
        - --requestheader-username-headers=X-Remote-User
        - --secure-port=6443
        - --service-account-key-file=/etc/kubernetes/pki/sa.pub
        - --service-cluster-ip-range=10.96.0.0/12
        - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
        - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
        image: k8s.gcr.io/kube-apiserver:v1.17.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 8
          httpGet:
            host: 172.200.1.200
            path: /healthz
            port: 6443
            scheme: HTTPS
          initialDelaySeconds: 15
          timeoutSeconds: 15
        name: kube-apiserver
        resources:
          requests:
            cpu: 250m
        volumeMounts:
        - mountPath: /etc/ssl/certs
          name: ca-certs
          readOnly: true
        - mountPath: /etc/pki
          name: etc-pki
          readOnly: true
        - mountPath: /etc/kubernetes/pki
          name: k8s-certs
          readOnly: true
      hostNetwork: true
      priorityClassName: system-cluster-critical
      volumes:
      - hostPath:
          path: /etc/ssl/certs
          type: DirectoryOrCreate
        name: ca-certs
      - hostPath:
          path: /etc/pki
          type: DirectoryOrCreate
        name: etc-pki
      - hostPath:
          path: /etc/kubernetes/pki
          type: DirectoryOrCreate
        name: k8s-certs
    status: {}
[dryrun] Would write file "/etc/kubernetes/manifests/kube-controller-manager.yaml" with content:
    apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: null
      labels:
        component: kube-controller-manager
        tier: control-plane
      name: kube-controller-manager
      namespace: kube-system
    spec:
      containers:
      - command:
        - kube-controller-manager
        - --allocate-node-cidrs=true
        - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
        - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
        - --bind-address=127.0.0.1
        - --client-ca-file=/etc/kubernetes/pki/ca.crt
        - --cluster-cidr=10.244.0.0/16
        - --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
        - --cluster-signing-key-file=/etc/kubernetes/pki/ca.key
        - --controllers=*,bootstrapsigner,tokencleaner
        - --kubeconfig=/etc/kubernetes/controller-manager.conf
        - --leader-elect=true
        - --node-cidr-mask-size=24
        - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
        - --root-ca-file=/etc/kubernetes/pki/ca.crt
        - --service-account-private-key-file=/etc/kubernetes/pki/sa.key
        - --service-cluster-ip-range=10.96.0.0/12
        - --use-service-account-credentials=true
        image: k8s.gcr.io/kube-controller-manager:v1.17.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 8
          httpGet:
            host: 127.0.0.1
            path: /healthz
            port: 10257
            scheme: HTTPS
          initialDelaySeconds: 15
          timeoutSeconds: 15
        name: kube-controller-manager
        resources:
          requests:
            cpu: 200m
        volumeMounts:
        - mountPath: /etc/ssl/certs
          name: ca-certs
          readOnly: true
        - mountPath: /etc/pki
          name: etc-pki
          readOnly: true
        - mountPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
          name: flexvolume-dir
        - mountPath: /etc/kubernetes/pki
          name: k8s-certs
          readOnly: true
        - mountPath: /etc/kubernetes/controller-manager.conf
          name: kubeconfig
          readOnly: true
      hostNetwork: true
      priorityClassName: system-cluster-critical
      volumes:
      - hostPath:
          path: /etc/ssl/certs
          type: DirectoryOrCreate
        name: ca-certs
      - hostPath:
          path: /etc/pki
          type: DirectoryOrCreate
        name: etc-pki
      - hostPath:
          path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
          type: DirectoryOrCreate
        name: flexvolume-dir
      - hostPath:
          path: /etc/kubernetes/pki
          type: DirectoryOrCreate
        name: k8s-certs
      - hostPath:
          path: /etc/kubernetes/controller-manager.conf
          type: FileOrCreate
        name: kubeconfig
    status: {}
[dryrun] Would write file "/etc/kubernetes/manifests/kube-scheduler.yaml" with content:
    apiVersion: v1
    kind: Pod
    metadata:
      creationTimestamp: null
      labels:
        component: kube-scheduler
        tier: control-plane
      name: kube-scheduler
      namespace: kube-system
    spec:
      containers:
      - command:
        - kube-scheduler
        - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
        - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
        - --bind-address=127.0.0.1
        - --kubeconfig=/etc/kubernetes/scheduler.conf
        - --leader-elect=true
        image: k8s.gcr.io/kube-scheduler:v1.17.2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 8
          httpGet:
            host: 127.0.0.1
            path: /healthz
            port: 10259
            scheme: HTTPS
          initialDelaySeconds: 15
          timeoutSeconds: 15
        name: kube-scheduler
        resources:
          requests:
            cpu: 100m
        volumeMounts:
        - mountPath: /etc/kubernetes/scheduler.conf
          name: kubeconfig
          readOnly: true
      hostNetwork: true
      priorityClassName: system-cluster-critical
      volumes:
      - hostPath:
          path: /etc/kubernetes/scheduler.conf
          type: FileOrCreate
        name: kubeconfig
    status: {}
[dryrun] Would write file "/var/lib/kubelet/config.yaml" with content:
    apiVersion: kubelet.config.k8s.io/v1beta1
    authentication:
      anonymous:
        enabled: false
      webhook:
        cacheTTL: 0s
        enabled: true
      x509:
        clientCAFile: /etc/kubernetes/pki/ca.crt
    authorization:
      mode: Webhook
      webhook:
        cacheAuthorizedTTL: 0s
        cacheUnauthorizedTTL: 0s
    clusterDNS:
    - 10.96.0.10
    clusterDomain: cluster.local
    cpuManagerReconcilePeriod: 0s
    evictionPressureTransitionPeriod: 0s
    fileCheckFrequency: 0s
    healthzBindAddress: 127.0.0.1
    healthzPort: 10248
    httpCheckFrequency: 0s
    imageMinimumGCAge: 0s
    kind: KubeletConfiguration
    nodeStatusReportFrequency: 0s
    nodeStatusUpdateFrequency: 0s
    rotateCertificates: true
    runtimeRequestTimeout: 0s
    staticPodPath: /etc/kubernetes/manifests
    streamingConnectionIdleTimeout: 0s
    syncFrequency: 0s
    volumeStatsAggPeriod: 0s
[dryrun] Would write file "/var/lib/kubelet/kubeadm-flags.env" with content:
    KUBELET_KUBEADM_ARGS="--cgroup-driver=cgroupfs --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.1"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/tmp/kubeadm-init-dryrun988930826". This can take up to 4m0s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[dryrun] Would perform action CREATE on resource "configmaps" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      ClusterConfiguration: |
        apiServer:
          extraArgs:
            authorization-mode: Node,RBAC
          timeoutForControlPlane: 4m0s
        apiVersion: kubeadm.k8s.io/v1beta2
        certificatesDir: /etc/kubernetes/pki
        clusterName: kubernetes
        controllerManager: {}
        dns:
          type: CoreDNS
        etcd:
          local:
            dataDir: /var/lib/etcd
        imageRepository: k8s.gcr.io
        kind: ClusterConfiguration
        kubernetesVersion: v1.17.2
        networking:
          dnsDomain: cluster.local
          podSubnet: 10.244.0.0/16
          serviceSubnet: 10.96.0.0/12
        scheduler: {}
      ClusterStatus: |
        apiEndpoints:
          master200.yinzhengjie.org.cn:
            advertiseAddress: 172.200.1.200
            bindPort: 6443
        apiVersion: kubeadm.k8s.io/v1beta2
        kind: ClusterStatus
    kind: ConfigMap
    metadata:
      creationTimestamp: null
      name: kubeadm-config
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "roles" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      creationTimestamp: null
      name: kubeadm:nodes-kubeadm-config
      namespace: kube-system
    rules:
    - apiGroups:
      - ""
      resourceNames:
      - kubeadm-config
      resources:
      - configmaps
      verbs:
      - get
[dryrun] Would perform action CREATE on resource "rolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:nodes-kubeadm-config
      namespace: kube-system
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: kubeadm:nodes-kubeadm-config
    subjects:
    - kind: Group
      name: system:bootstrappers:kubeadm:default-node-token
    - kind: Group
      name: system:nodes
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
[dryrun] Would perform action CREATE on resource "configmaps" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      kubelet: |
        apiVersion: kubelet.config.k8s.io/v1beta1
        authentication:
          anonymous:
            enabled: false
          webhook:
            cacheTTL: 0s
            enabled: true
          x509:
            clientCAFile: /etc/kubernetes/pki/ca.crt
        authorization:
          mode: Webhook
          webhook:
            cacheAuthorizedTTL: 0s
            cacheUnauthorizedTTL: 0s
        clusterDNS:
        - 10.96.0.10
        clusterDomain: cluster.local
        cpuManagerReconcilePeriod: 0s
        evictionPressureTransitionPeriod: 0s
        fileCheckFrequency: 0s
        healthzBindAddress: 127.0.0.1
        healthzPort: 10248
        httpCheckFrequency: 0s
        imageMinimumGCAge: 0s
        kind: KubeletConfiguration
        nodeStatusReportFrequency: 0s
        nodeStatusUpdateFrequency: 0s
        rotateCertificates: true
        runtimeRequestTimeout: 0s
        staticPodPath: /etc/kubernetes/manifests
        streamingConnectionIdleTimeout: 0s
        syncFrequency: 0s
        volumeStatsAggPeriod: 0s
    kind: ConfigMap
    metadata:
      creationTimestamp: null
      name: kubelet-config-1.17
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "roles" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      creationTimestamp: null
      name: kubeadm:kubelet-config-1.17
      namespace: kube-system
    rules:
    - apiGroups:
      - ""
      resourceNames:
      - kubelet-config-1.17
      resources:
      - configmaps
      verbs:
      - get
[dryrun] Would perform action CREATE on resource "rolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:kubelet-config-1.17
      namespace: kube-system
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: kubeadm:kubelet-config-1.17
    subjects:
    - kind: Group
      name: system:nodes
    - kind: Group
      name: system:bootstrappers:kubeadm:default-node-token
[dryrun] Would perform action GET on resource "nodes" in API group "core/v1"
[dryrun] Resource name: "master200.yinzhengjie.org.cn"
[dryrun] Would perform action PATCH on resource "nodes" in API group "core/v1"
[dryrun] Resource name: "master200.yinzhengjie.org.cn"
[dryrun] Attached patch:
    {"metadata":{"annotations":{"kubeadm.alpha.kubernetes.io/cri-socket":"/var/run/dockershim.sock"}}}
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master200.yinzhengjie.org.cn as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node master200.yinzhengjie.org.cn as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[dryrun] Would perform action GET on resource "nodes" in API group "core/v1"
[dryrun] Resource name: "master200.yinzhengjie.org.cn"
[dryrun] Would perform action PATCH on resource "nodes" in API group "core/v1"
[dryrun] Resource name: "master200.yinzhengjie.org.cn"
[dryrun] Attached patch:
    {"metadata":{"labels":{"node-role.kubernetes.io/master":""}},"spec":{"taints":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"}]}}
[bootstrap-token] Using token: dp9kc3.s7ceexx1xblkqklh
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[dryrun] Would perform action GET on resource "secrets" in API group "core/v1"
[dryrun] Resource name: "bootstrap-token-dp9kc3"
[dryrun] Would perform action CREATE on resource "secrets" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      auth-extra-groups: c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=
      description: VGhlIGRlZmF1bHQgYm9vdHN0cmFwIHRva2VuIGdlbmVyYXRlZCBieSAna3ViZWFkbSBpbml0Jy4=
      expiration: MjAyMC0wMi0wNVQxOTozNzozMCswODowMA==
      token-id: ZHA5a2Mz
      token-secret: czdjZWV4eDF4YmxrcWtsaA==
      usage-bootstrap-authentication: dHJ1ZQ==
      usage-bootstrap-signing: dHJ1ZQ==
    kind: Secret
    metadata:
      creationTimestamp: null
      name: bootstrap-token-dp9kc3
      namespace: kube-system
    type: bootstrap.kubernetes.io/token
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[dryrun] Would perform action CREATE on resource "clusterrolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:kubelet-bootstrap
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:node-bootstrapper
    subjects:
    - kind: Group
      name: system:bootstrappers:kubeadm:default-node-token
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[dryrun] Would perform action CREATE on resource "clusterrolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:node-autoapprove-bootstrap
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:certificates.k8s.io:certificatesigningrequests:nodeclient
    subjects:
    - kind: Group
      name: system:bootstrappers:kubeadm:default-node-token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[dryrun] Would perform action CREATE on resource "clusterrolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:node-autoapprove-certificate-rotation
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
    subjects:
    - kind: Group
      name: system:nodes
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[dryrun] Would perform action CREATE on resource "configmaps" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      kubeconfig: |
        apiVersion: v1
        clusters:
        - cluster:
            certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01ESXdOREV4TXpjeU4xb1hEVE13TURJd01URXhNemN5TjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSn
VaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjgrCmhvL1ptZkNhSTJITDJEam5wZk9kT1l2QnNxQTF0YmtNT1NGKzdNOTFHQ1FWZE9STG5IbUFRZTZQRExIbjQ5TUEKbmVyemNLTnp6S203T0liNmdDRlFyRERjYUwvaUV0VWlFQWhPQXd3ZTIwTTAzQlFWblptY3VlcVFqbE1BSUVrdApNamRRT3lhYnluUkt5bHUwSXZPRHJaeUE5Mjc2dEFFSWNnZlFEVEUrU2hwSmU1aUIveDNXOTMwZGN1T3F1cVgvClZvRzkxcHVDdjNGSlF5aEdBSE91emhQMmtodmVuemdkZXEyc2RoUlVReEU4OXJVU294Q3VBS2Z3TjVONGVaR0QKbWRVc0dHS0pGblRLOXA5YkpFNncxWGh1d1VqTmltZmdhc2RvbVBOMmxVaU84MFFCcDdWM3ZlNjNhbVVYRVhVWQpVYk1oUFVlL21mZ1hyQnJZME1FQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFCMXdtWW9tYzBXN210aEVlK0ErK0lkTGVrOEoKRUwyNnkyVXJFR3g5MVRkeTRvZFpCcVVpRzltNERmdTNlZlU2T0g3V0gyTG1RNGNXbWtLcitDdmJrRThRK3FHMwo3TGRvNTdzbHhwN2NyYU5tQ3dmb1ZIOCtLbVl4UUlGOWpxMGJ4RjlqRlNMSnJHa0N6cDBwMlhaQi9Tc1VFb3M4ClNsU0hvWEhuemQ2ekZSTmJLKzYyMFkvV1ZrU2h1V3VuRW5rV1kva0hHdTB6eHJqOUJiQ25yM0FETzVqZ0tyM08KN24wdmhnZzlJaE1Rb25lYldJTEdNVXV1b21VNlBvWDN3eHprOVJkSjV6T05samN6bXhVc2IvUFJEcnQ3bkFGUQpjYmdGclcrOUo1VmkyMFNMLzZmK1ZXZnV3aHNpVXFaU1VLWE10dE9OQitjYkRlVFVqYjN3MHErSjNPbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=            server: https://172.200.1.200:6443
          name: ""
        contexts: null
        current-context: ""
        kind: Config
        preferences: {}
        users: null
    kind: ConfigMap
    metadata:
      creationTimestamp: null
      name: cluster-info
      namespace: kube-public
[dryrun] Would perform action CREATE on resource "roles" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      creationTimestamp: null
      name: kubeadm:bootstrap-signer-clusterinfo
      namespace: kube-public
    rules:
    - apiGroups:
      - ""
      resourceNames:
      - cluster-info
      resources:
      - configmaps
      verbs:
      - get
[dryrun] Would perform action CREATE on resource "rolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:bootstrap-signer-clusterinfo
      namespace: kube-public
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: kubeadm:bootstrap-signer-clusterinfo
    subjects:
    - kind: User
      name: system:anonymous
[dryrun] Would perform action GET on resource "configmaps" in API group "core/v1"
[dryrun] Resource name: "kube-dns"
[dryrun] Would perform action GET on resource "configmaps" in API group "core/v1"
[dryrun] Resource name: "coredns"
[dryrun] Would perform action CREATE on resource "configmaps" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      Corefile: |
        .:53 {
            errors
            health {
               lameduck 5s
            }
            ready
            kubernetes cluster.local in-addr.arpa ip6.arpa {
               pods insecure
               fallthrough in-addr.arpa ip6.arpa
               ttl 30
            }
            prometheus :9153
            forward . /etc/resolv.conf
            cache 30
            loop
            reload
            loadbalance
        }
    kind: ConfigMap
    metadata:
      creationTimestamp: null
      name: coredns
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "clusterroles" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      creationTimestamp: null
      name: system:coredns
    rules:
    - apiGroups:
      - ""
      resources:
      - endpoints
      - services
      - pods
      - namespaces
      verbs:
      - list
      - watch
    - apiGroups:
      - ""
      resources:
      - nodes
      verbs:
      - get
[dryrun] Would perform action CREATE on resource "clusterrolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      creationTimestamp: null
      name: system:coredns
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:coredns
    subjects:
    - kind: ServiceAccount
      name: coredns
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "serviceaccounts" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      creationTimestamp: null
      name: coredns
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "deployments" in API group "apps/v1"
[dryrun] Attached object:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: kube-dns
      name: coredns
      namespace: kube-system
    spec:
      replicas: 2
      selector:
        matchLabels:
          k8s-app: kube-dns
      strategy:
        rollingUpdate:
          maxUnavailable: 1
        type: RollingUpdate
      template:
        metadata:
          creationTimestamp: null
          labels:
            k8s-app: kube-dns
        spec:
          containers:
          - args:
            - -conf
            - /etc/coredns/Corefile
            image: k8s.gcr.io/coredns:1.6.5
            imagePullPolicy: IfNotPresent
            livenessProbe:
              failureThreshold: 5
              httpGet:
                path: /health
                port: 8080
                scheme: HTTP
              initialDelaySeconds: 60
              successThreshold: 1
              timeoutSeconds: 5
            name: coredns
            ports:
            - containerPort: 53
              name: dns
              protocol: UDP
            - containerPort: 53
              name: dns-tcp
              protocol: TCP
            - containerPort: 9153
              name: metrics
              protocol: TCP
            readinessProbe:
              httpGet:
                path: /ready
                port: 8181
                scheme: HTTP
            resources:
              limits:
                memory: 170Mi
              requests:
                cpu: 100m
                memory: 70Mi
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                add:
                - NET_BIND_SERVICE
                drop:
                - all
              readOnlyRootFilesystem: true
            volumeMounts:
            - mountPath: /etc/coredns
              name: config-volume
              readOnly: true
          dnsPolicy: Default
          nodeSelector:
            beta.kubernetes.io/os: linux
          priorityClassName: system-cluster-critical
          serviceAccountName: coredns
          tolerations:
          - key: CriticalAddonsOnly
            operator: Exists
          - effect: NoSchedule
            key: node-role.kubernetes.io/master
          volumes:
          - configMap:
              items:
              - key: Corefile
                path: Corefile
              name: coredns
            name: config-volume
    status: {}
[dryrun] Would perform action CREATE on resource "services" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        prometheus.io/port: "9153"
        prometheus.io/scrape: "true"
      creationTimestamp: null
      labels:
        k8s-app: kube-dns
        kubernetes.io/cluster-service: "true"
        kubernetes.io/name: KubeDNS
      name: kube-dns
      namespace: kube-system
      resourceVersion: "0"
    spec:
      clusterIP: 10.96.0.10
      ports:
      - name: dns
        port: 53
        protocol: UDP
        targetPort: 53
      - name: dns-tcp
        port: 53
        protocol: TCP
        targetPort: 53
      - name: metrics
        port: 9153
        protocol: TCP
        targetPort: 9153
      selector:
        k8s-app: kube-dns
    status:
      loadBalancer: {}
[addons] Applied essential addon: CoreDNS
[dryrun] Would perform action CREATE on resource "serviceaccounts" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      creationTimestamp: null
      name: kube-proxy
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "configmaps" in API group "core/v1"
[dryrun] Attached object:
    apiVersion: v1
    data:
      config.conf: |-
        apiVersion: kubeproxy.config.k8s.io/v1alpha1
        bindAddress: 0.0.0.0
        clientConnection:
          acceptContentTypes: ""
          burst: 0
          contentType: ""
          kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
          qps: 0
        clusterCIDR: 10.244.0.0/16
        configSyncPeriod: 0s
        conntrack:
          maxPerCore: null
          min: null
          tcpCloseWaitTimeout: null
          tcpEstablishedTimeout: null
        enableProfiling: false
        healthzBindAddress: ""
        hostnameOverride: ""
        iptables:
          masqueradeAll: false
          masqueradeBit: null
          minSyncPeriod: 0s
          syncPeriod: 0s
        ipvs:
          excludeCIDRs: null
          minSyncPeriod: 0s
          scheduler: ""
          strictARP: false
          syncPeriod: 0s
        kind: KubeProxyConfiguration
        metricsBindAddress: ""
        mode: ""
        nodePortAddresses: null
        oomScoreAdj: null
        portRange: ""
        udpIdleTimeout: 0s
        winkernel:
          enableDSR: false
          networkName: ""
          sourceVip: ""
      kubeconfig.conf: |-
        apiVersion: v1
        kind: Config
        clusters:
        - cluster:
            certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
            server: https://172.200.1.200:6443
          name: default
        contexts:
        - context:
            cluster: default
            namespace: default
            user: default
          name: default
        current-context: default
        users:
        - name: default
          user:
            tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    kind: ConfigMap
    metadata:
      creationTimestamp: null
      labels:
        app: kube-proxy
      name: kube-proxy
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "daemonsets" in API group "apps/v1"
[dryrun] Attached object:
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: kube-proxy
      name: kube-proxy
      namespace: kube-system
    spec:
      selector:
        matchLabels:
          k8s-app: kube-proxy
      template:
        metadata:
          creationTimestamp: null
          labels:
            k8s-app: kube-proxy
        spec:
          containers:
          - command:
            - /usr/local/bin/kube-proxy
            - --config=/var/lib/kube-proxy/config.conf
            - --hostname-override=$(NODE_NAME)
            env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            image: k8s.gcr.io/kube-proxy:v1.17.2
            imagePullPolicy: IfNotPresent
            name: kube-proxy
            resources: {}
            securityContext:
              privileged: true
            volumeMounts:
            - mountPath: /var/lib/kube-proxy
              name: kube-proxy
            - mountPath: /run/xtables.lock
              name: xtables-lock
            - mountPath: /lib/modules
              name: lib-modules
              readOnly: true
          hostNetwork: true
          nodeSelector:
            beta.kubernetes.io/os: linux
          priorityClassName: system-node-critical
          serviceAccountName: kube-proxy
          tolerations:
          - key: CriticalAddonsOnly
            operator: Exists
          - operator: Exists
          volumes:
          - configMap:
              name: kube-proxy
            name: kube-proxy
          - hostPath:
              path: /run/xtables.lock
              type: FileOrCreate
            name: xtables-lock
          - hostPath:
              path: /lib/modules
            name: lib-modules
      updateStrategy:
        type: RollingUpdate
    status:
      currentNumberScheduled: 0
      desiredNumberScheduled: 0
      numberMisscheduled: 0
      numberReady: 0
[dryrun] Would perform action CREATE on resource "clusterrolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      creationTimestamp: null
      name: kubeadm:node-proxier
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:node-proxier
    subjects:
    - kind: ServiceAccount
      name: kube-proxy
      namespace: kube-system
[dryrun] Would perform action CREATE on resource "roles" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
      creationTimestamp: null
      name: kube-proxy
      namespace: kube-system
    rules:
    - apiGroups:
      - ""
      resourceNames:
      - kube-proxy
      resources:
      - configmaps
      verbs:
      - get
[dryrun] Would perform action CREATE on resource "rolebindings" in API group "rbac.authorization.k8s.io/v1"
[dryrun] Attached object:
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
      creationTimestamp: null
      name: kube-proxy
      namespace: kube-system
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: kube-proxy
    subjects:
    - kind: Group
      name: system:bootstrappers:kubeadm:default-node-token
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/tmp/kubeadm-init-dryrun988930826/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.200.1.200:6443 --token dp9kc3.s7ceexx1xblkqklh 
    --discovery-token-ca-cert-hash sha256:c87a7b40890a61c0b9c23d71418f1b63987a74d943839b018eb836d1cdded4eb 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# kubeadm init --kubernetes-version="v1.17.2" --pod-network-cidr="10.244.0.0/16" --dry-run        #使用"--dry-run"表示测试运行,如果没有报错将该参数去掉即可

[root@master200.yinzhengjie.org.cn ~]# kubeadm init --kubernetes-version="v1.17.2" --pod-network-cidr="10.244.0.0/16" 
W0204 19:39:12.076223   32177 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0204 19:39:12.076260   32177 validation.go:28] Cannot validate kubelet config - no validator is available
[init] Using Kubernetes version: v1.17.2
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [master200.yinzhengjie.org.cn kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 172.200.1.200]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [master200.yinzhengjie.org.cn localhost] and IPs [172.200.1.200 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [master200.yinzhengjie.org.cn localhost] and IPs [172.200.1.200 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0204 19:39:14.609238   32177 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0204 19:39:14.609827   32177 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 18.505316 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master200.yinzhengjie.org.cn as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node master200.yinzhengjie.org.cn as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: gu02ed.9k1rpyl4mtkhvdpk
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk 
    --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# echo $?
0
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# kubeadm init --kubernetes-version="v1.17.2" --pod-network-cidr="10.244.0.0/16"              #初始化集群,注意k8s版本要和咱们下载的版本指定一致哟~

6>.部署网络插件

[root@master200.yinzhengjie.org.cn ~]# mkdir .kube
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# cp /etc/kubernetes/admin.conf .kube/config
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get nodes        #kubectl是k8s的apiserver的客户端工具,我们可以通过该命令查看节点的状态,很明显是"NotReady"状态,我们需要执行吓一跳命令下载网络插件
NAME                           STATUS     ROLES    AGE     VERSION
master200.yinzhengjie.org.cn   NotReady   master   6m21s   v1.17.2
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml        #下载flannel网络插件
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n kube-system      #使用"-n"查看名称"kube-system"的名称空间,如下图所示,所有组件都是"Running"状态,因此我们的master节点应该也是"Ready"状态啦~
NAME                                                   READY   STATUS    RESTARTS   AGE
coredns-6955765f44-455fh                               0/1     Running   0          8m17s
coredns-6955765f44-q6zqj                               0/1     Running   0          8m17s
etcd-master200.yinzhengjie.org.cn                      1/1     Running   0          8m31s
kube-apiserver-master200.yinzhengjie.org.cn            1/1     Running   0          8m31s
kube-controller-manager-master200.yinzhengjie.org.cn   1/1     Running   0          8m31s
kube-flannel-ds-amd64-hnnhb                            1/1     Running   0          33s
kube-proxy-6r9dx                                       1/1     Running   0          8m17s
kube-scheduler-master200.yinzhengjie.org.cn            1/1     Running   0          8m31s
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# kubectl get nodes              #安装flannel(由CoreOS公司研发)网络插件后,它会自动运行,之后咱们的k8s master节点的状态成功变为"Ready"。
NAME                           STATUS   ROLES    AGE     VERSION
master200.yinzhengjie.org.cn   Ready    master   9m40s   v1.17.2
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

7>.将k8s master节点的镜像打包并发送到k8s node节点

[root@master200.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                v1.17.2             cba2a99699bd        2 weeks ago         116MB
k8s.gcr.io/kube-apiserver            v1.17.2             41ef50a5f06a        2 weeks ago         171MB
k8s.gcr.io/kube-controller-manager   v1.17.2             da5fd66c4068        2 weeks ago         161MB
k8s.gcr.io/kube-scheduler            v1.17.2             f52d4c527ef2        2 weeks ago         94.4MB
k8s.gcr.io/coredns                   1.6.5               70f311871ae1        3 months ago        41.6MB
k8s.gcr.io/etcd                      3.4.3-0             303ce5db0e90        3 months ago        288MB
quay.io/coreos/flannel               v0.11.0-amd64       ff281650a721        12 months ago       52.6MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        2 years ago         742kB
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# ll
total 0
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# docker image save k8s.gcr.io/kube-proxy:v1.17.2 k8s.gcr.io/pause:3.1 quay.io/coreos/flannel:v0.11.0-amd64 -o k8s-node-v1.17.2.tar
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# ll
total 170028
-rw------- 1 root root 174108672 Feb  4 20:18 k8s-node-v1.17.2.tar
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# ll -h
total 167M
-rw------- 1 root root 167M Feb  4 20:18 k8s-node-v1.17.2.tar
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp k8s-node-v1.17.2.tar node201.yinzhengjie.org.cn:~
root@node201.yinzhengjie.org.cn's password: 
k8s-node-v1.17.2.tar                                                                                                 100%  166MB 119.7MB/s   00:01    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp k8s-node-v1.17.2.tar node202.yinzhengjie.org.cn:~
root@node202.yinzhengjie.org.cn's password: 
k8s-node-v1.17.2.tar                                                                                                                                                100%  166MB 118.3MB/s   00:01    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp k8s-node-v1.17.2.tar node203.yinzhengjie.org.cn:~
root@node203.yinzhengjie.org.cn's password: 
k8s-node-v1.17.2.tar                                                                                                                                                 100%  166MB 122.9MB/s   00:01    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

8>.各个k8s node节点导入镜像

[root@node201.yinzhengjie.org.cn ~]# ll
total 170028
-rw------- 1 root root 174108672 Feb  4 20:19 k8s-node-v1.17.2.tar
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar 
7bff100f35cb: Loading layer [==================================================>]  4.672MB/4.672MB
5d3f68f6da8f: Loading layer [==================================================>]  9.526MB/9.526MB
9b48060f404d: Loading layer [==================================================>]  5.912MB/5.912MB
3f3a4ce2b719: Loading layer [==================================================>]  35.25MB/35.25MB
9ce0bb155166: Loading layer [==================================================>]   5.12kB/5.12kB
Loaded image: quay.io/coreos/flannel:v0.11.0-amd64
fc4976bd934b: Loading layer [==================================================>]  53.88MB/53.88MB
682fbb19de80: Loading layer [==================================================>]  21.06MB/21.06MB
2dc2f2423ad1: Loading layer [==================================================>]  5.168MB/5.168MB
ad9fb2411669: Loading layer [==================================================>]  4.608kB/4.608kB
597151d24476: Loading layer [==================================================>]  8.192kB/8.192kB
0d8d54147a3a: Loading layer [==================================================>]  8.704kB/8.704kB
ca7fe3329548: Loading layer [==================================================>]  37.81MB/37.81MB
Loaded image: k8s.gcr.io/kube-proxy:v1.17.2
e17133b79956: Loading layer [==================================================>]  744.4kB/744.4kB
Loaded image: k8s.gcr.io/pause:3.1
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy    v1.17.2             cba2a99699bd        2 weeks ago         116MB
quay.io/coreos/flannel   v0.11.0-amd64       ff281650a721        12 months ago       52.6MB
k8s.gcr.io/pause         3.1                 da86e6ba6ca1        2 years ago         742kB
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar

[root@node202.yinzhengjie.org.cn ~]# ll
total 170028
-rw------- 1 root root 174108672 Feb  4 20:20 k8s-node-v1.17.2.tar
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar 
7bff100f35cb: Loading layer [==================================================>]  4.672MB/4.672MB
5d3f68f6da8f: Loading layer [==================================================>]  9.526MB/9.526MB
9b48060f404d: Loading layer [==================================================>]  5.912MB/5.912MB
3f3a4ce2b719: Loading layer [==================================================>]  35.25MB/35.25MB
9ce0bb155166: Loading layer [==================================================>]   5.12kB/5.12kB
Loaded image: quay.io/coreos/flannel:v0.11.0-amd64
fc4976bd934b: Loading layer [==================================================>]  53.88MB/53.88MB
682fbb19de80: Loading layer [==================================================>]  21.06MB/21.06MB
2dc2f2423ad1: Loading layer [==================================================>]  5.168MB/5.168MB
ad9fb2411669: Loading layer [==================================================>]  4.608kB/4.608kB
597151d24476: Loading layer [==================================================>]  8.192kB/8.192kB
0d8d54147a3a: Loading layer [==================================================>]  8.704kB/8.704kB
ca7fe3329548: Loading layer [==================================================>]  37.81MB/37.81MB
Loaded image: k8s.gcr.io/kube-proxy:v1.17.2
e17133b79956: Loading layer [==================================================>]  744.4kB/744.4kB
Loaded image: k8s.gcr.io/pause:3.1
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# ll
total 170028
-rw------- 1 root root 174108672 Feb  4 20:20 k8s-node-v1.17.2.tar
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy    v1.17.2             cba2a99699bd        2 weeks ago         116MB
quay.io/coreos/flannel   v0.11.0-amd64       ff281650a721        12 months ago       52.6MB
k8s.gcr.io/pause         3.1                 da86e6ba6ca1        2 years ago         742kB
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar

[root@node203.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# ll
total 170028
-rw------- 1 root root 174108672 Feb  4 20:20 k8s-node-v1.17.2.tar
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar 
7bff100f35cb: Loading layer [==================================================>]  4.672MB/4.672MB
5d3f68f6da8f: Loading layer [==================================================>]  9.526MB/9.526MB
9b48060f404d: Loading layer [==================================================>]  5.912MB/5.912MB
3f3a4ce2b719: Loading layer [==================================================>]  35.25MB/35.25MB
9ce0bb155166: Loading layer [==================================================>]   5.12kB/5.12kB
Loaded image: quay.io/coreos/flannel:v0.11.0-amd64
fc4976bd934b: Loading layer [==================================================>]  53.88MB/53.88MB
682fbb19de80: Loading layer [==================================================>]  21.06MB/21.06MB
2dc2f2423ad1: Loading layer [==================================================>]  5.168MB/5.168MB
ad9fb2411669: Loading layer [==================================================>]  4.608kB/4.608kB
597151d24476: Loading layer [==================================================>]  8.192kB/8.192kB
0d8d54147a3a: Loading layer [==================================================>]  8.704kB/8.704kB
ca7fe3329548: Loading layer [==================================================>]  37.81MB/37.81MB
Loaded image: k8s.gcr.io/kube-proxy:v1.17.2
e17133b79956: Loading layer [==================================================>]  744.4kB/744.4kB
Loaded image: k8s.gcr.io/pause:3.1
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# docker image ls
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy    v1.17.2             cba2a99699bd        2 weeks ago         116MB
quay.io/coreos/flannel   v0.11.0-amd64       ff281650a721        12 months ago       52.6MB
k8s.gcr.io/pause         3.1                 da86e6ba6ca1        2 years ago         742kB
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# docker image load -i k8s-node-v1.17.2.tar

四.将k8s node节点加入到上一步初始化的k8s集群

1>.将k8s master节点的k8s软件源拷贝到其它3个k8s node节点

[root@master200.yinzhengjie.org.cn ~]# scp /etc/yum.repos.d/kubernetes.repo node201.yinzhengjie.org.cn:/etc/yum.repos.d/kubernetes.repo
root@node201.yinzhengjie.org.cn's password: 
kubernetes.repo                                                                                                                                                                                                                             100%  275   125.6KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /etc/yum.repos.d/kubernetes.repo node202.yinzhengjie.org.cn:/etc/yum.repos.d/kubernetes.repo
root@node202.yinzhengjie.org.cn's password: 
kubernetes.repo                                                                                                                                                                                                                             100%  275   199.6KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /etc/yum.repos.d/kubernetes.repo node203.yinzhengjie.org.cn:/etc/yum.repos.d/kubernetes.repo
root@node203.yinzhengjie.org.cn's password: 
kubernetes.repo                                                                                                                                                                                                                             100%  275    63.0KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

2>.三个节点均安装k8s相关软件包

[root@node201.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
kubernetes/primary                                                                                                                                                                                                                                     |  63 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                                            460/460
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: kubectl >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
---> Package kubelet.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.17.2-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.17.2-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubectl.x86_64 0:1.17.2-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                   Arch                                                      Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 kubeadm                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    8.7 M
 kubelet                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                     20 M
Installing for dependencies:
 conntrack-tools                                                           x86_64                                                    1.4.4-5.el7_7.2                                                      updates                                                       187 k
 cri-tools                                                                 x86_64                                                    1.13.0-0                                                             kubernetes                                                    5.1 M
 kubectl                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    9.4 M
 kubernetes-cni                                                            x86_64                                                    0.7.5-0                                                              kubernetes                                                     10 M
 libnetfilter_cthelper                                                     x86_64                                                    1.0.0-10.el7_7.1                                                     updates                                                        18 k
 libnetfilter_cttimeout                                                    x86_64                                                    1.0.0-6.el7_7.1                                                      updates                                                        18 k
 libnetfilter_queue                                                        x86_64                                                    1.0.2-2.el7_2                                                        base                                                           23 k
 socat                                                                     x86_64                                                    1.7.3.2-2.el7                                                        base                                                          290 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  2 Packages (+8 Dependent packages)

Total download size: 54 M
Installed size: 243 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-5.el7_7.2.x86_64.rpm                                                                                                                                                                                                     | 187 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY                                 ] 3.5 MB/s | 6.9 MB  00:00:13 ETA 
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                                                                                                                                 | 5.1 MB  00:00:01     
(3/10): 105d89f0607c7baf91305ba352e78000bd20aad5cdf706bffff3b31cd546dbf3-kubeadm-1.17.2-0.x86_64.rpm                                                                                                                                                   | 8.7 MB  00:00:02     
(4/10): b44630896c69cd411db53be1d5cb5ae899a40aba7c0766317ea904390fcfc45b-kubectl-1.17.2-0.x86_64.rpm                                                                                                                                                   | 9.4 MB  00:00:01     
(5/10): libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:00     
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                                                                                                                                    |  23 kB  00:00:00     
(7/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                                                                                                                             |  10 MB  00:00:01     
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                                                                                                                                 | 290 kB  00:00:01     
(9/10): libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:02     
(10/10): 3ee7f2dff78e6fbb3ac3af8acb1a907f4bec1b1ef4cf627cbe02fa553707f2e9-kubelet-1.17.2-0.x86_64.rpm                                                                                                                                                  |  20 MB  00:00:04     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         8.6 MB/s |  54 MB  00:00:06     
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                2/10 
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 3/10 
  Installing : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   5/10 
  Installing : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             6/10 
  Installing : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    7/10 
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             8/10 
  Installing : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Installing : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                  10/10 
  Verifying  : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Verifying  : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    2/10 
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   3/10 
  Verifying  : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 5/10 
  Verifying  : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                   6/10 
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             7/10 
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                8/10 
  Verifying  : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Verifying  : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                            10/10 

Installed:
  kubeadm.x86_64 0:1.17.2-0                                                                                                             kubelet.x86_64 0:1.17.2-0                                                                                                            

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 cri-tools.x86_64 0:1.13.0-0 kubectl.x86_64 0:1.17.2-0 kubernetes-cni.x86_64 0:0.7.5-0 libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2
  socat.x86_64 0:1.7.3.2-2.el7            

Complete!
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm

[root@node202.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.bit.edu.cn
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
kubernetes/primary                                                                                                                                                                                                                                     |  63 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                                            460/460
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: kubectl >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
---> Package kubelet.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.17.2-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.17.2-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubectl.x86_64 0:1.17.2-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                   Arch                                                      Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 kubeadm                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    8.7 M
 kubelet                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                     20 M
Installing for dependencies:
 conntrack-tools                                                           x86_64                                                    1.4.4-5.el7_7.2                                                      updates                                                       187 k
 cri-tools                                                                 x86_64                                                    1.13.0-0                                                             kubernetes                                                    5.1 M
 kubectl                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    9.4 M
 kubernetes-cni                                                            x86_64                                                    0.7.5-0                                                              kubernetes                                                     10 M
 libnetfilter_cthelper                                                     x86_64                                                    1.0.0-10.el7_7.1                                                     updates                                                        18 k
 libnetfilter_cttimeout                                                    x86_64                                                    1.0.0-6.el7_7.1                                                      updates                                                        18 k
 libnetfilter_queue                                                        x86_64                                                    1.0.2-2.el7_2                                                        base                                                           23 k
 socat                                                                     x86_64                                                    1.7.3.2-2.el7                                                        base                                                          290 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  2 Packages (+8 Dependent packages)

Total download size: 54 M
Installed size: 243 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-5.el7_7.2.x86_64.rpm                                                                                                                                                                                                     | 187 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY                                 ] 1.8 MB/s | 3.7 MB  00:00:28 ETA 
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                                                                                                                                 | 5.1 MB  00:00:01     
(3/10): b44630896c69cd411db53be1d5cb5ae899a40aba7c0766317ea904390fcfc45b-kubectl-1.17.2-0.x86_64.rpm                                                                                                                                                   | 9.4 MB  00:00:01     
(4/10): 3ee7f2dff78e6fbb3ac3af8acb1a907f4bec1b1ef4cf627cbe02fa553707f2e9-kubelet-1.17.2-0.x86_64.rpm                                                                                                                                                   |  20 MB  00:00:02     
(5/10): libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:00     
(6/10): libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:00     
(7/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                                                                                                                                    |  23 kB  00:00:00     
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                                                                                                                                 | 290 kB  00:00:00     
(9/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                                                                                                                             |  10 MB  00:00:01     
(10/10): 105d89f0607c7baf91305ba352e78000bd20aad5cdf706bffff3b31cd546dbf3-kubeadm-1.17.2-0.x86_64.rpm                                                                                                                                                  | 8.7 MB  00:00:18     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         2.9 MB/s |  54 MB  00:00:18     
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                2/10 
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 3/10 
  Installing : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   5/10 
  Installing : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             6/10 
  Installing : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    7/10 
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             8/10 
  Installing : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Installing : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                  10/10 
  Verifying  : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Verifying  : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    2/10 
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   3/10 
  Verifying  : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 5/10 
  Verifying  : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                   6/10 
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             7/10 
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                8/10 
  Verifying  : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Verifying  : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                            10/10 

Installed:
  kubeadm.x86_64 0:1.17.2-0                                                                                                             kubelet.x86_64 0:1.17.2-0                                                                                                            

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 cri-tools.x86_64 0:1.13.0-0 kubectl.x86_64 0:1.17.2-0 kubernetes-cni.x86_64 0:0.7.5-0 libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2
  socat.x86_64 0:1.7.3.2-2.el7            

Complete!
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm

[root@node203.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
kubernetes/primary                                                                                                                                                                                                                                     |  63 kB  00:00:01     
kubernetes                                                                                                                                                                                                                                                            460/460
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: kubectl >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.17.2-0.x86_64
---> Package kubelet.x86_64 0:1.17.2-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.17.2-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.17.2-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-5.el7_7.2.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubectl.x86_64 0:1.17.2-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================================================================================================================================================================================================
 Package                                                                   Arch                                                      Version                                                              Repository                                                     Size
==============================================================================================================================================================================================================================================================================
Installing:
 kubeadm                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    8.7 M
 kubelet                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                     20 M
Installing for dependencies:
 conntrack-tools                                                           x86_64                                                    1.4.4-5.el7_7.2                                                      updates                                                       187 k
 cri-tools                                                                 x86_64                                                    1.13.0-0                                                             kubernetes                                                    5.1 M
 kubectl                                                                   x86_64                                                    1.17.2-0                                                             kubernetes                                                    9.4 M
 kubernetes-cni                                                            x86_64                                                    0.7.5-0                                                              kubernetes                                                     10 M
 libnetfilter_cthelper                                                     x86_64                                                    1.0.0-10.el7_7.1                                                     updates                                                        18 k
 libnetfilter_cttimeout                                                    x86_64                                                    1.0.0-6.el7_7.1                                                      updates                                                        18 k
 libnetfilter_queue                                                        x86_64                                                    1.0.2-2.el7_2                                                        base                                                           23 k
 socat                                                                     x86_64                                                    1.7.3.2-2.el7                                                        base                                                          290 k

Transaction Summary
==============================================================================================================================================================================================================================================================================
Install  2 Packages (+8 Dependent packages)

Total download size: 54 M
Installed size: 243 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-5.el7_7.2.x86_64.rpm                                                                                                                                                                                                     | 187 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY                                 ] 1.4 MB/s | 7.3 MB  00:00:34 ETA 
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                                                                                                                                 | 5.1 MB  00:00:03     
(3/10): 105d89f0607c7baf91305ba352e78000bd20aad5cdf706bffff3b31cd546dbf3-kubeadm-1.17.2-0.x86_64.rpm                                                                                                                                                   | 8.7 MB  00:00:03     
(4/10): b44630896c69cd411db53be1d5cb5ae899a40aba7c0766317ea904390fcfc45b-kubectl-1.17.2-0.x86_64.rpm                                                                                                                                                   | 9.4 MB  00:00:02     
(5/10): libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:00     
(6/10): libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64.rpm                                                                                                                                                                                              |  18 kB  00:00:00     
(7/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                                                                                                                                    |  23 kB  00:00:00     
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                                                                                                                                 | 290 kB  00:00:00     
(9/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                                                                                                                             |  10 MB  00:00:03     
(10/10): 3ee7f2dff78e6fbb3ac3af8acb1a907f4bec1b1ef4cf627cbe02fa553707f2e9-kubelet-1.17.2-0.x86_64.rpm                                                                                                                                                  |  20 MB  00:00:06     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                                                         5.3 MB/s |  54 MB  00:00:10     
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
Retrieving key from https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                2/10 
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 3/10 
  Installing : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   5/10 
  Installing : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             6/10 
  Installing : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    7/10 
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             8/10 
  Installing : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Installing : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                  10/10 
  Verifying  : libnetfilter_cthelper-1.0.0-10.el7_7.1.x86_64                                                                                                                                                                                                             1/10 
  Verifying  : conntrack-tools-1.4.4-5.el7_7.2.x86_64                                                                                                                                                                                                                    2/10 
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                                                   3/10 
  Verifying  : kubectl-1.17.2-0.x86_64                                                                                                                                                                                                                                   4/10 
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                                                                 5/10 
  Verifying  : kubeadm-1.17.2-0.x86_64                                                                                                                                                                                                                                   6/10 
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                                                             7/10 
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                                                                8/10 
  Verifying  : kubelet-1.17.2-0.x86_64                                                                                                                                                                                                                                   9/10 
  Verifying  : libnetfilter_cttimeout-1.0.0-6.el7_7.1.x86_64                                                                                                                                                                                                            10/10 

Installed:
  kubeadm.x86_64 0:1.17.2-0                                                                                                             kubelet.x86_64 0:1.17.2-0                                                                                                            

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-5.el7_7.2 cri-tools.x86_64 0:1.13.0-0 kubectl.x86_64 0:1.17.2-0 kubernetes-cni.x86_64 0:0.7.5-0 libnetfilter_cthelper.x86_64 0:1.0.0-10.el7_7.1 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7_7.1 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2
  socat.x86_64 0:1.7.3.2-2.el7            

Complete!
[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# yum -y install kubelet kubeadm

[root@node201.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               disabled
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               enabled 
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# systemctl enable kubelet.service        #这里别忘记把kubulet服务设置为开机自启动,如果不设置其实也没有多大关系,只不过我们将k8s node加入k8s集群时可能会提示抛出警告信息哟~

[root@node202.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               disabled
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               enabled 
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# systemctl enable kubelet.service

[root@node203.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               disabled
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# systemctl list-unit-files | grep kubelet
kubelet.service                               enabled 
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# systemctl enable kubelet.service

3>.将k8s master节点的"/etc/sysconfig/kubelet"拷贝到其它三个k8s node节点(如果你的节点已经禁用了swap分区,此步骤可直接忽略)

[root@master200.yinzhengjie.org.cn ~]# cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS="--fail-swap-on=false"
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /etc/sysconfig/kubelet node201.yinzhengjie.org.cn:/etc/sysconfig/kubelet
root@node201.yinzhengjie.org.cn's password: 
kubelet                                                                                                                                                                                                                                     100%   42    23.3KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /etc/sysconfig/kubelet node202.yinzhengjie.org.cn:/etc/sysconfig/kubelet
root@node202.yinzhengjie.org.cn's password: 
kubelet                                                                                                                                                                                                                                     100%   42    32.2KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]# scp /etc/sysconfig/kubelet node203.yinzhengjie.org.cn:/etc/sysconfig/kubelet
root@node203.yinzhengjie.org.cn's password: 
kubelet                                                                                                                                                                                                                                     100%   42    21.7KB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

4>.各节点加入到k8s集群

[root@node201.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk 
>     --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794 
W0204 20:11:11.564558   32762 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# echo $?
0
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk \ > --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794

[root@node202.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794 
W0204 20:26:07.049488   31891 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# echo $?
0
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794

[root@node203.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794 
W0204 20:26:19.457993   32429 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
    [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# kubeadm join 172.200.1.200:6443 --token gu02ed.9k1rpyl4mtkhvdpk --discovery-token-ca-cert-hash sha256:755ff7d07588b75aec769e566e4257d832cf6d1ec79bcdbd655dad80f30a6794

5>.将节点加入成功后,再来k8s master节点查看节点是否加入成功

[root@master200.yinzhengjie.org.cn ~]# kubectl get nodes              #如果k8s master和k8s node状态的状态为"Ready"说明k8s集群搭建完毕啦~
NAME                           STATUS   ROLES    AGE   VERSION
master200.yinzhengjie.org.cn   Ready    master   47m   v1.17.2
node201.yinzhengjie.org.cn     Ready    <none>   16m   v1.17.2
node202.yinzhengjie.org.cn     Ready    <none>   78s   v1.17.2
node203.yinzhengjie.org.cn     Ready    <none>   66s   v1.17.2
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

五.配置k8s node节点也可以管理api server服务器

1>.在各个k8s node节点创建"~/.kubu"目录

[root@node201.yinzhengjie.org.cn ~]# mkdir ~/.kube
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# ll .kube/
total 0
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]#

[root@node201.yinzhengjie.org.cn ~]# mkdir ~/.kube

[root@node202.yinzhengjie.org.cn ~]# mkdir .kube
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]# ll .kube/
total 0
[root@node202.yinzhengjie.org.cn ~]# 
[root@node202.yinzhengjie.org.cn ~]#

[root@node202.yinzhengjie.org.cn ~]# mkdir .kube

[root@node203.yinzhengjie.org.cn ~]# mkdir ~/.kube
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]# ll .kube/
total 0
[root@node203.yinzhengjie.org.cn ~]# 
[root@node203.yinzhengjie.org.cn ~]#

[root@node203.yinzhengjie.org.cn ~]# mkdir ~/.kube

2>.将k8s master节点的"/etc/kubernetes/admin.conf"配置文件拷贝到各个k8s node节点的"~/.kube/"目录并命名为"config"

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node201.yinzhengjie.org.cn:~/.kube/config
root@node201.yinzhengjie.org.cn's password: 
admin.conf                                                                              100% 5453     2.2MB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node201.yinzhengjie.org.cn:~/.kube/config

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node202.yinzhengjie.org.cn:~/.kube/config
root@node202.yinzhengjie.org.cn's password: 
admin.conf                                                                              100% 5453     3.6MB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node202.yinzhengjie.org.cn:~/.kube/config

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node203.yinzhengjie.org.cn:~/.kube/config
root@node203.yinzhengjie.org.cn's password: 
admin.conf                                                                              100% 5453     2.9MB/s   00:00    
[root@master200.yinzhengjie.org.cn ~]# 
[root@master200.yinzhengjie.org.cn ~]#

[root@master200.yinzhengjie.org.cn ~]# scp /etc/kubernetes/admin.conf node203.yinzhengjie.org.cn:~/.kube/config

3>.在k8s node节点查看node信息

[root@node201.yinzhengjie.org.cn ~]# kubectl get nodes
NAME                           STATUS   ROLES    AGE   VERSION
master200.yinzhengjie.org.cn   Ready    master   61m   v1.17.2
node201.yinzhengjie.org.cn     Ready    <none>   29m   v1.17.2
node202.yinzhengjie.org.cn     Ready    <none>   15m   v1.17.2
node203.yinzhengjie.org.cn     Ready    <none>   14m   v1.17.2
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://172.200.1.200:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
[root@node201.yinzhengjie.org.cn ~]# 
[root@node201.yinzhengjie.org.cn ~]#

4>.API Server的证书说明

  早期版本的K8S的API Server支持2个套接字,分别监听两个不同的端口,一个是8080(http协议),另一个是6443(https),后来官方考虑http并不安全于是采用了只保留6443端口。  

  API Server是需要双向认证的,即除了API Server有自己的证书文件还要求客户端也提供证书文件,这就是为什么上面会将"/etc/kubernetes/admin.conf"拷贝到"~/.kube/config"的原因。  

  我们使用kubeadmin部署K8S集群时,API Server组件需要的证书会自动帮咱们生成,默认存放目录在"/etc/kubernetes/pki/",如下图所示。

5>.kubernetes快速入门

  博主推荐阅读:
    https://developer.aliyun.com/article/1604377
相关实践学习
通过Ingress进行灰度发布
本场景您将运行一个简单的应用,部署一个新的应用用于新的发布,并通过Ingress能力实现灰度发布。
容器应用与集群管理
欢迎来到《容器应用与集群管理》课程,本课程是“云原生容器Clouder认证“系列中的第二阶段。课程将向您介绍与容器集群相关的概念和技术,这些概念和技术可以帮助您了解阿里云容器服务ACK/ACK Serverless的使用。同时,本课程也会向您介绍可以采取的工具、方法和可操作步骤,以帮助您了解如何基于容器服务ACK Serverless构建和管理企业级应用。 学习完本课程后,您将能够: 掌握容器集群、容器编排的基本概念 掌握Kubernetes的基础概念及核心思想 掌握阿里云容器服务ACK/ACK Serverless概念及使用方法 基于容器服务ACK Serverless搭建和管理企业级网站应用
目录
相关文章
|
22天前
|
缓存 自然语言处理 并行计算
基于NVIDIA A30 加速卡推理部署通义千问-72B-Chat测试过程
本文介绍了基于阿里云通义千问72B大模型(Qwen-72B-Chat)的性能基准测试,包括测试环境准备、模型部署、API测试等内容。测试环境配置为32核128G内存的ECS云主机,配备8块NVIDIA A30 GPU加速卡。软件环境包括Ubuntu 22.04、CUDA 12.4.0、PyTorch 2.4.0等。详细介绍了模型下载、部署命令及常见问题解决方法,并展示了API测试结果和性能分析。
860 1
|
1月前
|
机器学习/深度学习 监控 计算机视觉
目标检测实战(八): 使用YOLOv7完成对图像的目标检测任务(从数据准备到训练测试部署的完整流程)
本文介绍了如何使用YOLOv7进行目标检测,包括环境搭建、数据集准备、模型训练、验证、测试以及常见错误的解决方法。YOLOv7以其高效性能和准确率在目标检测领域受到关注,适用于自动驾驶、安防监控等场景。文中提供了源码和论文链接,以及详细的步骤说明,适合深度学习实践者参考。
333 0
目标检测实战(八): 使用YOLOv7完成对图像的目标检测任务(从数据准备到训练测试部署的完整流程)
|
1月前
|
机器学习/深度学习 并行计算 数据可视化
目标分类笔记(二): 利用PaddleClas的框架来完成多标签分类任务(从数据准备到训练测试部署的完整流程)
这篇文章介绍了如何使用PaddleClas框架完成多标签分类任务,包括数据准备、环境搭建、模型训练、预测、评估等完整流程。
95 0
目标分类笔记(二): 利用PaddleClas的框架来完成多标签分类任务(从数据准备到训练测试部署的完整流程)
|
1月前
|
机器学习/深度学习 数据采集 算法
目标分类笔记(一): 利用包含多个网络多种训练策略的框架来完成多目标分类任务(从数据准备到训练测试部署的完整流程)
这篇博客文章介绍了如何使用包含多个网络和多种训练策略的框架来完成多目标分类任务,涵盖了从数据准备到训练、测试和部署的完整流程,并提供了相关代码和配置文件。
51 0
目标分类笔记(一): 利用包含多个网络多种训练策略的框架来完成多目标分类任务(从数据准备到训练测试部署的完整流程)
|
1月前
|
机器学习/深度学习 XML 并行计算
目标检测实战(七): 使用YOLOX完成对图像的目标检测任务(从数据准备到训练测试部署的完整流程)
这篇文章介绍了如何使用YOLOX完成图像目标检测任务的完整流程,包括数据准备、模型训练、验证和测试。
168 0
目标检测实战(七): 使用YOLOX完成对图像的目标检测任务(从数据准备到训练测试部署的完整流程)
|
运维 Kubernetes Linux
kubernetes(2):Centos7 安装 kubernetes 1.5 版本
1,kubernetes 安装 centos 7 已经集成了kubernetes 1.5 直接安装就可以了。 非常的简单,方便。 yum install kubernetes etcd docker 同时会,安装一个kubernetes-client kubernetes-master kubernetes-node 运维也是非常的方便了。1.5.2-
2540 0
|
Kubernetes Linux Docker
kubernetes(1):Centos7 安装 kubernetes 1.4.0 版本
首先看到最新的kubernetes 1.4.0 发布了。激动了下,就开始研究了。 本文的原文连接是: http://blog.csdn.net/freewebsys/article/details/52685592 未经博主允许不得转载。 博主地址是:http://blog.csdn.net/freewebsys 1,关于k8s Kubernetes
1844 0
|
5天前
|
Kubernetes 监控 Cloud Native
Kubernetes集群的高可用性与伸缩性实践
Kubernetes集群的高可用性与伸缩性实践
24 1
|
26天前
|
JSON Kubernetes 容灾
ACK One应用分发上线:高效管理多集群应用
ACK One应用分发上线,主要介绍了新能力的使用场景
|
27天前
|
Kubernetes 持续交付 开发工具
ACK One GitOps:ApplicationSet UI简化多集群GitOps应用管理
ACK One GitOps新发布了多集群应用控制台,支持管理Argo CD ApplicationSet,提升大规模应用和集群的多集群GitOps应用分发管理体验。
下一篇
无影云桌面