方式二---配置文件初始化:
此文件名称为kubeadm-init.yaml,可以通过kubeadm 命令生成模板文件,命令为:
kubeadm config print init-defaults > kubeadm-init.yaml
此模板文件需要修改8个地方:
- ttl: 24h0m0s 修改为ttl: "0" 这样初始化的token不会过期
- advertiseAddress: 1.2.3.4 修改为advertiseAddress: 192.168.217.19 这个IP是master节点的IP
- name;node 修改为name:master 也就是修改为master节点的主机名,我的master节点的主机名是master
- dns: {} 修改为dns: type: CoreDNS 指定集群的DNS类型
- imageRepository: k8s.gcr.io修改为阿里云的镜像站点---registry.aliyuncs.com/google_containers,这样可以提高下载速度,也就是镜像的本地化
- podSubnet: "10.244.0.0/16" 这个是增加的,原模板文件里没有这个,等同于设置apiserver里的--pod-network-cidr,这个网段是pod使用的。
- serviceSubnet: "" 这里可以不设置,默认就是10.96.0.0/12 此网段是service这个资源使用的。
- kubernetesVersion: 1.22.2 这个是kubeadm,kubelet的版本号
版本号的查询:
[root@master ~]# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:37:34Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
初始化config配置文件示例---kubeadm-init.yaml:
apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: "0" usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.217.19 bindPort: 6443 nodeRegistration: criSocket: /var/run/dockershim.sock imagePullPolicy: IfNotPresent name: master taints: null --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: type: CoreDNS etcd: local: dataDir: /var/lib/etcds imageRepository: registry.aliyuncs.com/google_containers kind: ClusterConfiguration kubernetesVersion: 1.22.2 networking: dnsDomain: cluster.local podSubnet: "10.244.0.0/16" serviceSubnet: "" scheduler: {}
使用配置文件初始化集群:
kubeadm init --config=kubeadm-init.yaml
工作节点的加入:
方式一---命令行增加工作节点在20和21节点,执行此命令:
kubeadm join 192.168.217.19:6443 --token b1zldq.89t1aea8szja9d7l \ --discovery-token-ca-cert-hash sha256:6ac4ccaf392e4173b7fd9c09cebfd0e2d7eb5ff5a826f39409701fe012ad2ba4
此命令输出如下:
[root@node1 ~]# kubeadm join 192.168.217.19:6443 --token b1zldq.89t1aea8szja9d7l \ > --discovery-token-ca-cert-hash sha256:6ac4ccaf392e4173b7fd9c09cebfd0e2d7eb5ff5a826f39409701fe012ad2ba4 [preflight] Running pre-flight checks [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service' [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml' [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... This node has joined the cluster: * Certificate signing request was sent to apiserver and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
方式二---kubeadm config 配置文件方式增加工作节点
生成join工作节点的模板文件:
kubeadm config print join-defaults >kubeadm-join.yaml
编辑文件kubeadm-join.yaml,如下地方需要修改:
- apiServerEndpoint:连接apiserver的地址,即master的api地址,这里可以改为192.168.0.1:6443,如果master集群部署的话,这里需要改为集群vip地址
- token及tlsBootstrapToken:连接master使用的token,这里需要与master上的InitConfiguration中的token配置一致
- name:node节点的名称,如果使用主机名,需要确保master节点可以解析该主机名。否则的话可直接使用ip地址
本例中的kubeadm-join.yaml
token和tlsBootstrapToken 的值都是上面init命令输出的最后一行的token,此文件在工作节点运行,运行命令为:
kubeadm join --config=kubeadm-join.yaml
示例文件内容如下:
apiVersion: kubeadm.k8s.io/v1beta3 caCertPath: /etc/kubernetes/pki/ca.crt kind: JoinConfiguration discovery: bootstrapToken: apiServerEndpoint: 192.168.217.19:6443 token: b1zldq.89t1aea8szja9d7l unsafeSkipCAVerification: true t1sBootstrapToken: b1zldq.89t1aea8szja9d7l
五,极为简单的网络插件部署
在主节点master执行:
kubectl apply -f kube-flannel.yml chmod a+x flannel cp flannel /opt/cni/bin/ scp flannel node1:/opt/cni/bin/ scp flannel node2:/opt/cni/bin/
在三个节点都重启kubelet服务:
systemctl restart kubelet
此时,在master节点查询节点状态:
[root@master ~]# kubectl get no NAME STATUS ROLES AGE VERSION master Ready control-plane,master 26m v1.22.2 node1 Ready <none> 25m v1.22.2 node2 Ready <none> 25m v1.22.2
六,集群的一个小bug修复及集群的功能测试
小bug修复:
编辑文件/etc/kubernetes/manifests/kube-controller-manager.yaml 删除- --port=0 这一行 编辑文件/etc/kubernetes/manifests/kube-scheduler.yaml 删除- --port=0 这一行 重启kubelet服务:systemctl restart kubelet
看看集群的健康状态:
[root@master ~]# kubectl get cs Warning: v1 ComponentStatus is deprecated in v1.19+ NAME STATUS MESSAGE ERROR controller-manager Healthy ok etcd-0 Healthy {"health":"true","reason":""} scheduler Healthy ok
看看集群的pod是否正常,service的clusterIP是否正常:
[root@master ~]# kubectl get po,svc -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/coredns-7f6cbbb7b8-dcnpf 1/1 Running 0 109m kube-system pod/coredns-7f6cbbb7b8-hg5t8 1/1 Running 0 109m kube-system pod/etcd-master 1/1 Running 0 109m kube-system pod/kube-apiserver-master 1/1 Running 0 109m kube-system pod/kube-controller-manager-master 1/1 Running 0 56s kube-system pod/kube-flannel-ds-22k8b 1/1 Running 0 94m kube-system pod/kube-flannel-ds-mgvsj 1/1 Running 0 94m kube-system pod/kube-flannel-ds-v8ml5 1/1 Running 0 94m kube-system pod/kube-proxy-hstwd 1/1 Running 0 107m kube-system pod/kube-proxy-sqmfq 1/1 Running 0 107m kube-system pod/kube-proxy-z2cmx 1/1 Running 0 109m kube-system pod/kube-scheduler-master 1/1 Running 0 111s NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 109m kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 109m
生成一个nginx的pod,看此pod能否正常部署:
[root@master ~]# kubectl create deploy nginx --image=nginx:1.20 deployment.apps/nginx created [root@master ~]# kubectl get po NAME READY STATUS RESTARTS AGE nginx-7fb9867-ssqsr 0/1 ContainerCreating 0 9s [root@master ~]# kubectl get po NAME READY STATUS RESTARTS AGE nginx-7fb9867-ssqsr 0/1 ContainerCreating 0 11s
总结:
此次实践需要指出的是,这种方式部署的kubernetes集群是只能做测试用的,因为,etcd只是单例,不是高可用集群,apiserver也不是ha高可用。后续会给出一个可用于生产的高可用kubeadm版本集群。
附:
在线安装kubeadm方式部署的kubernetes集群:
cat > /etc/yum.repos.d/kubernetes.repo << EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
其它的步骤基本一样,没有什么需要改的,只是在线的方式会比较慢,因为镜像需要一个个慢慢下载,前面的初始化集群命令那的kubernetes的版本需要更改,例如:
yum 安装的是kubernetes-1.23.9:
yum install -y kubeadm-1.23.9 kubelet-1.23.9 kubectl-1.23.9 conntrack-tools libseccomp \ libtool-ltdl device-mapper-persistent-data lvm2
那么,初始化命令需要修改版本号为:
kubeadm init \ --apiserver-advertise-address=192.168.217.19 \ --image-repository registry.aliyuncs.com/google_containers \ --kubernetes-version v1.23.9 \ --service-cidr=10.96.0.0/12 \ --pod-network-cidr=10.244.0.0/16