1、kubernetes集群部署
- 注：若没有
特别指明操作的节点，默认所有操作均在k8s-01节点中执行 - kubernetes master 节点运行组件：etcd、kube-apiserver、kube-controller-manager、kube-scheduler
- kubernetes node 节点运行组件：docker、flannel、kubelet、kube-proxy、coredns、dashboard
- 因为master也当node节点使用，所以所有节点都部署了docker和flannel
1.0、创建CA证书和秘钥
- 为确保安全，kubernetes各个组件需要使用x509证书对通信进行加密和认证
- CA(Certificate Authority)是自签名的根证书，用来签名后续创建的其他证书
- 使用CloudFlare的PKI工具cfssl创建所有证书
1.0.0、安装cfssl工具
k8s-01:~ # cd /opt/k8s/packages/ k8s-01:/opt/k8s/packages # wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 k8s-01:/opt/k8s/packages # wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 k8s-01:/opt/k8s/packages # wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
k8s-01:/opt/k8s/packages # mv cfssl_linux-amd64 /opt/k8s/bin/cfssl k8s-01:/opt/k8s/packages # mv cfssljson_linux-amd64 /opt/k8s/bin/cfssljson k8s-01:/opt/k8s/packages # mv cfssl-certinfo_linux-amd64 /opt/k8s/bin/cfssl-certinfo k8s-01:/opt/k8s/packages # chmod +x /opt/k8s/bin/*
1.0.1、创建根证书
k8s-01:~ # cd /opt/k8s/ssl/ k8s-01:/opt/k8s/ssl # cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "usages": [ "signing", "key encipherment", "server auth", "client auth" ], "expiry": "876000h" } } } } EOF
signing表示该证书可用于签名其它证书，生成的ca.pem证书找中CA=TRUEserver auth表示client可以用该证书对server提供的证书进行验证
client auth表示server可以用该证书对client提供的证书进行验证
1.0.2、创建证书签名请求文件
k8s-01:/opt/k8s/ssl # cat > ca-csr.json <<EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "ShangHai", "L": "ShangHai", "O": "k8s", "OU": "bandian" } ], "ca": { "expiry": "876000h" } } EOF
CN:CommonNamekube-apiserver从证书中提取该字段作为请求的用户名(User Name)，浏览器使用该字段验证网站是否合法O:Organizationkube-apiserver从证书中提取该字段作为请求的用户和所属组(Group)
kube-apiserver将提取的User、Group作为RBAC授权的用户和标识
1.0.3、生成CA证书和秘钥
k8s-01:/opt/k8s/ssl # cfssl gencert -initca ca-csr.json | cfssljson -bare ca
1.0.4、分发CA证书到所有节点
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${NODE_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" scp /opt/k8s/ssl/{ca*.pem,ca-config.json} ${host}:/etc/kubernetes/cert done
1.1、部署kubectl命令
kubectl默认从~/.kube/config读取kube-apiserver地址和认证信息
"kubernetes二进制包的github网址，包含了kubernetes-1.9到kubernetes-1.21所有版本" https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG "将下载好的包，上传到k8s-01的/opt/k8s/packages目录下，解压即可"
k8s-01:~ # cd /opt/k8s/packages/ k8s-01:/opt/k8s/packages # tar xf kubernetes-server-linux-amd64.tar.gz
1.1.0、分发kubectl命令到所有节点
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${NODE_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" scp /opt/k8s/packages/kubernetes/server/bin/kubectl ${host}:/opt/k8s/bin ssh root@${host} "kubectl completion bash > /etc/bash_completion.d/kubectl" done
kubectl completion bash > /etc/bash_completion.d/kubectl 配置kubectl命令自动补全，依赖bash-completion，如果没有，需要先安装bash-completion（suse一般都自带，centos发行版需要执行yum -y install bash-completion）
1.1.1、创建admin证书和秘钥
kubectl作为集群的管理工具，需要被授予最高权限，这里创建具有最高权限的admin证书kubectl与apiserver进行https通信，apiserver对提供的证书进行认证和授权
k8s-01:~ # cd /opt/k8s/ssl/ k8s-01:/opt/k8s/ssl # cat > admin-csr.json <<EOF { "CN": "admin", "hosts": [ ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "ShangHai", "L": "ShangHai", "O": "system:masters", "OU": "bandian" } ] } EOF
- O 为system:masters，kube-apiserver收到该证书后将请求的Group设置为system:masters
- 预定的ClusterRoleBinding cluster-admin将Group system:masters与Role cluster-admin绑定，该Role授予API的权限
- 该证书只有被kubectl当做client证书使用，所以hosts字段为空
1.1.2、生成admin证书和秘钥
k8s-01:/opt/k8s/ssl # cfssl gencert -ca=/opt/k8s/ssl/ca.pem \ -ca-key=/opt/k8s/ssl/ca-key.pem \ -config=/opt/k8s/ssl/ca-config.json \ -profile=kubernetes admin-csr.json | cfssljson -bare admin
1.1.3、创建kubeconfig文件
k8s-01:/opt/k8s/ssl # source /opt/k8s/bin/k8s-env.sh
"设置集群参数" k8s-01:/opt/k8s/ssl # kubectl config set-cluster kubernetes \ --certificate-authority=/opt/k8s/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=kubectl.kubeconfig
"设置客户端认证参数" k8s-01:/opt/k8s/ssl # kubectl config set-credentials admin \ --client-certificate=/opt/k8s/ssl/admin.pem \ --client-key=/opt/k8s/ssl/admin-key.pem \ --embed-certs=true \ --kubeconfig=kubectl.kubeconfig
"设置上下文参数" k8s-01:/opt/k8s/ssl # kubectl config set-context kubernetes \ --cluster=kubernetes \ --user=admin \ --kubeconfig=kubectl.kubeconfig
"设置默认上下文" k8s-01:/opt/k8s/ssl # kubectl config use-context kubernetes --kubeconfig=kubectl.kubeconfig
--certificate-authority验证kube-apiserver证书的根证书--client-certificate、--client-key刚生成的admin证书和私钥，连接kube-apiserver时使用--embed-certs=true将ca.pem和admin.pem证书嵌入到生成的kubectl.kubeconfig文件中 (如果不加入，写入的是证书文件路径，后续拷贝kubeconfig到其它机器时，还需要单独拷贝证书)
1.1.4、分发kubeconfig文件
- 分发到使用kubectl命令的节点（一般在master上管理）
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${MASTER_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" ssh root@${host} "mkdir ~/.kube" scp /opt/k8s/ssl/kubectl.kubeconfig ${host}:~/.kube/config done
