絮叨
本来想着直接用Rancher直接搞K8s的，但是我想着因为是学习还是有一个学习过程，就想着走一套官方的一个搭建过程，并且熟悉一下人家的设计思想。
Kubernetes是什么？
Kubernetes是容器集群管理系统，是一个开源的平台，可以实现容器集群的自动化部署、自动扩缩容、维护等功能。 通过Kubernetes你可以：
- 快速部署应用
- 快速扩展应用
- 无缝对接新的应用功能
- 节省资源，优化硬件资源的使用
Kubernetes 特点
- 可移植: 支持公有云，私有云，混合云，多重云（multi-cloud）
- 可扩展: 模块化, 插件化, 可挂载, 可组合
- 自动化: 自动部署，自动重启，自动复制，自动伸缩/扩展
使用Kubernetes能做什么？
- 多个进程（作为容器运行）协同工作。（Pod）
- 存储系统挂载
- Distributing secrets
- 应用健康检测
- 应用实例的复制
- Pod自动伸缩/扩展
- Naming and discovering
- 负载均衡
- 滚动更新
- 资源监控
- 日志访问
- 调试应用程序
- 提供认证和授权
Kubernetes 组件
K8s是由许多个的组件组成的 分为Master组件和节点（Node）组件
Master组件
Master组件提供集群的管理控制中心。 Master组件可以在集群中任何节点上运行。但是为了简单起见，通常在一台VM/机器上启动所有Master组件，并且不会在此VM/机器上运行用户容器。
kube-apiserver
kube-apiserver用于暴露Kubernetes API。任何的资源请求/调用操作都是通过kube-apiserver提供的接口进行。请参阅构建高可用群集。
ETCD
etcd是Kubernetes提供默认的存储系统，保存所有集群数据，使用时需要为etcd数据提供备份计划。
kube-controller-manager
kube-controller-manager运行管理控制器，它们是集群中处理常规任务的后台线程。逻辑上，每个控制器是一个单独的进程，但为了降低复杂性，它们都被编译成单个二进制文件，并在单个进程中运行。
kube-scheduler
kube-scheduler 监视新创建没有分配到Node的Pod，为Pod选择一个Node。
节点（Node）组件
节点组件运行在Node，提供Kubernetes运行时环境，以及维护Pod。
kubelet
kubelet是主要的节点代理，它会监视已分配给节点的pod，具体功能：
- 安装Pod所需的volume。
- 下载Pod的Secrets。
- Pod中运行的 docker（或experimentally，rkt）容器。
- 定期执行容器健康检查。
- Reports the status of the pod back to the rest of the system, by creating a mirror pod if necessary.
- Reports the status of the node back to the rest of the system.
kube-proxy
kube-proxy通过在主机上维护网络规则并执行连接转发来实现Kubernetes服务抽象。
Kubernetes 安装前的准备
概述
本次安装采用 Ubuntu Server X64 16.04 LTS 版本安装 kubernetes 集群环境，集群节点为 1 主 2 从模式，此次对虚拟机会有些基本要求，如下：
- OS：Ubuntu Server X64 18.04 LTS（16.04 版本步骤相同，再之前则不同）
- CPU：最低要求，1 CPU 2 核
- 内存：最低要求，2GB
- 磁盘：最低要求，20GB
创建三台虚拟机，分别命名如下：
- Ubuntu Server 18.04 X64 Kubernetes Master
- Ubuntu Server 18.04 X64 Kubernetes Slave1
- Ubuntu Server 18.04 X64 Kubernetes Slave2
对虚拟机系统的配置：
- 关闭交换空间：sudo swapoff -a
- 避免开机启动交换空间：注释 /etc/fstab 中的 swap
- 关闭防火墙：ufw disable
使用 APT 安装 Docker
安装
# 更新软件源 sudo apt-get update # 安装所需依赖 sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common # 安装 GPG 证书 curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add - # 新增软件源信息 sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" # 再次更新软件源 sudo apt-get -y update # 安装 Docker CE 版 sudo apt-get -y install docker-ce 复制代码
验证
docker version Client: Version: 18.09.6 API version: 1.39 Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 02:35:57 2019 OS/Arch: linux/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 18.09.6 API version: 1.39 (minimum version 1.12) Go version: go1.10.8 Git commit: 481bc77 Built: Sat May 4 01:59:36 2019 OS/Arch: linux/amd64 Experimental: false 复制代码
配置加速器
对于使用 systemd 的系统，请在 /etc/docker/daemon.json 中写入如下内容（如果文件不存在请新建该文件）
{ "registry-mirrors": [ "https://registry.docker-cn.com" ] } 复制代码
验证加速器是否配置成功：
sudo systemctl restart docker docker info ... # 出现如下语句即表示配置成功 Registry Mirrors: https://registry.docker-cn.com/ ... 复制代码
修改主机名
在同一局域网中主机名不应该相同，所以我们需要做修改，直接修改 /etc/hostname 里的名称即可
vim /etc/hostname #kubernetes-master 复制代码
安装 kubeadm
kubeadm 是 kubernetes 的集群安装工具，能够快速安装 kubernetes 集群。
配置软件源
# 安装系统工具 apt-get update && apt-get install -y apt-transport-https # 安装 GPG 证书 curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - # 写入软件源；注意：我们用系统代号为 bionic，但目前阿里云不支持，所以沿用 16.04 的 xenial cat << EOF >/etc/apt/sources.list.d/kubernetes.list > deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main > EOF 复制代码
安装 kubeadm，kubelet，kubectl
# 安装 apt-get update apt-get install -y kubelet kubeadm kubectl # 安装过程如下，注意 kubeadm 的版本号 Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: conntrack cri-tools kubernetes-cni socat The following NEW packages will be installed: conntrack cri-tools kubeadm kubectl kubelet kubernetes-cni socat 0 upgraded, 7 newly installed, 0 to remove and 96 not upgraded. Need to get 50.6 MB of archives. After this operation, 290 MB of additional disk space will be used. Get:1 http://mirrors.aliyun.com/ubuntu bionic/main amd64 conntrack amd64 1:1.4.4+snapshot20161117-6ubuntu2 [30.6 kB] Get:2 http://mirrors.aliyun.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] Get:3 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 cri-tools amd64 1.12.0-00 [5,343 kB] Get:4 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubernetes-cni amd64 0.7.5-00 [6,473 kB] Get:5 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubelet amd64 1.14.1-00 [21.5 MB] Get:6 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubectl amd64 1.14.1-00 [8,806 kB] Get:7 https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial/main amd64 kubeadm amd64 1.14.1-00 [8,150 kB] Fetched 50.6 MB in 5s (9,912 kB/s) Selecting previously unselected package conntrack. (Reading database ... 67205 files and directories currently installed.) Preparing to unpack .../0-conntrack_1%3a1.4.4+snapshot20161117-6ubuntu2_amd64.deb ... Unpacking conntrack (1:1.4.4+snapshot20161117-6ubuntu2) ... Selecting previously unselected package cri-tools. Preparing to unpack .../1-cri-tools_1.12.0-00_amd64.deb ... Unpacking cri-tools (1.12.0-00) ... Selecting previously unselected package kubernetes-cni. Preparing to unpack .../2-kubernetes-cni_0.7.5-00_amd64.deb ... Unpacking kubernetes-cni (0.7.5-00) ... Selecting previously unselected package socat. Preparing to unpack .../3-socat_1.7.3.2-2ubuntu2_amd64.deb ... Unpacking socat (1.7.3.2-2ubuntu2) ... Selecting previously unselected package kubelet. Preparing to unpack .../4-kubelet_1.14.1-00_amd64.deb ... Unpacking kubelet (1.14.1-00) ... Selecting previously unselected package kubectl. Preparing to unpack .../5-kubectl_1.14.1-00_amd64.deb ... Unpacking kubectl (1.14.1-00) ... Selecting previously unselected package kubeadm. Preparing to unpack .../6-kubeadm_1.14.1-00_amd64.deb ... Unpacking kubeadm (1.14.1-00) ... Setting up conntrack (1:1.4.4+snapshot20161117-6ubuntu2) ... Setting up kubernetes-cni (0.7.5-00) ... Setting up cri-tools (1.12.0-00) ... Setting up socat (1.7.3.2-2ubuntu2) ... Setting up kubelet (1.14.1-00) ... Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /lib/systemd/system/kubelet.service. Setting up kubectl (1.14.1-00) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... # 注意这里的版本号，我们使用的是 kubernetes v1.14.1 Setting up kubeadm (1.14.1-00) ... # 设置 kubelet 自启动，并启动 kubelet systemctl enable kubelet && systemctl start kubelet 复制代码
- kubeadm：用于初始化 Kubernetes 集群
- kubectl：Kubernetes 的命令行工具，主要作用是部署和管理应用，查看各种资源，创建，删除和更新组件
- kubelet：主要负责启动 Pod 和容器
配置 kubeadm
安装 kubernetes 主要是安装它的各个镜像，而 kubeadm 已经为我们集成好了运行 kubernetes 所需的基本镜像。但由于国内的网络原因，在搭建环境时，无法拉取到这些镜像。此时我们只需要修改为阿里云提供的镜像服务即可解决该问题
创建并修改配置
kubeadm config print init-defaults --kubeconfig ClusterConfiguration > kubeadm.yml 复制代码
修改配置内容
# 修改配置为如下内容 apiVersion: kubeadm.k8s.io/v1beta1 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: # 修改为主节点 IP advertiseAddress: 192.168.141.130 bindPort: 6443 nodeRegistration: criSocket: /var/run/dockershim.sock name: kubernetes-master taints: - effect: NoSchedule key: node-role.kubernetes.io/master --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta1 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controlPlaneEndpoint: "" controllerManager: {} dns: type: CoreDNS etcd: local: dataDir: /var/lib/etcd # 国内不能访问 Google，修改为阿里云 imageRepository: registry.aliyuncs.com/google_containers kind: ClusterConfiguration # 修改版本号 kubernetesVersion: v1.17.0 networking: dnsDomain: cluster.local # 配置成 Calico 的默认网段 podSubnet: "192.168.0.0/16" serviceSubnet: 10.96.0.0/12 scheduler: {} --- # 开启 IPVS 模式 apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration featureGates: SupportIPVSProxyMode: true mode: ipvs 复制代码
查看和拉取镜像
# 查看所需镜像列表 kubeadm config images list --config kubeadm.yml # 拉取镜像 kubeadm config images pull --config kubeadm.yml 复制代码
安装 kubernetes 主节点
执行以下命令初始化主节点，该命令指定了初始化时需要使用的配置文件，其中添加 --upload-certs 参数可以在后续执行加入节点时自动分发证书文件。追加的 tee kubeadm-init.log 用以输出日志。
kubeadm init --config=kubeadm.yml --experimental-upload-certs | tee kubeadm-init.log #your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version. W0106 17:26:39.941946 25748 common.go:77] your configuration file uses a deprecated API spec: "kubeadm.k8s.io/v1beta1". Please use 'kubeadm config migrate --old-config old.yaml --new-config new.yaml', which will write the new, similar spec using a newer API version. W0106 17:26:39.944487 25748 validation.go:28] Cannot validate kube-proxy config - no validator is available W0106 17:26:39.944533 25748 validation.go:28] Cannot validate kubelet config - no validator is available [init] Using Kubernetes version: v1.17.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kubernetes-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.62.159] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [kubernetes-master localhost] and IPs [192.168.62.159 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [kubernetes-master localhost] and IPs [192.168.62.159 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" W0106 17:26:44.531537 25748 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [control-plane] Creating static Pod manifest for "kube-controller-manager" W0106 17:26:44.532749 25748 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 24.504808 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace [upload-certs] Using certificate key: 781e69718ac47389e4e65f025cb9cb842ae621f495323b08f8c3cc5fe69c5a82 [mark-control-plane] Marking the node kubernetes-master as control-plane by adding the label "node-role.kubernetes.io/master=''" [mark-control-plane] Marking the node kubernetes-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] [bootstrap-token] Using token: abcdef.0123456789abcdef [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.62.159:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:7237dd082021214d77c1d99f0cdc2a1a110c33ba94c5e2df699ea3cebbab1ea4 复制代码
注意：如果安装 kubernetes 版本和下载的镜像版本不统一则会出现 timed out waiting for the condition 错误。中途失败或是想修改配置可以使用 kubeadm reset 命令重置配置，再做初始化操作即可。
配置 kubectl
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config(非root才做) 复制代码
验证是否成功
kubectl get node 复制代码
至此主节点配置完成
- kubeadm init 的执行过程
- init：指定版本进行初始化操作
- preflight：初始化前的检查和下载所需要的 Docker 镜像文件
- kubelet-start：生成 kubelet 的配置文件 var/lib/kubelet/config.yaml，没有这个文件 kubelet 无法启动，所以初始化之前的 - kubelet 实际上启动不会成功
- certificates：生成 Kubernetes 使用的证书，存放在 /etc/kubernetes/pki 目录中
- kubeconfig：生成 KubeConfig 文件，存放在 /etc/kubernetes 目录中，组件之间通信需要使用对应文件
- control-plane：使用 /etc/kubernetes/manifest 目录下的 YAML 文件，安装 Master 组件
- etcd：使用 /etc/kubernetes/manifest/etcd.yaml 安装 Etcd 服务
- wait-control-plane：等待 control-plan 部署的 Master 组件启动
- apiclient：检查 Master 组件服务状态。
- uploadconfig：更新配置
- kubelet：使用 configMap 配置 kubelet
- patchnode：更新 CNI 信息到 Node 上，通过注释的方式记录
- mark-control-plane：为当前节点打标签，打了角色 Master，和不可调度标签，这样默认就不会使用 Master 节点来运行 Pod
- bootstrap-token：生成 token 记录下来，后边使用 kubeadm join 往集群中添加节点时会用到
- addons：安装附加组件 CoreDNS 和 kube-proxy-
结尾
k8s的Master节点已经搭建完毕，下一节我们讲讲它的Node搭建（说实话K8s还是有点难度的，难搞哦，但是大家第一次玩不要问为啥，先把他用起来了之后，我们再反过来弄清楚它的为什么）
