一文教会你,如何通过kubeadm,在生产环境部署K8S高可用集群(二)

简介: 一文教会你,如何通过kubeadm,在生产环境部署K8S高可用集群(二)

一文教会你,如何通过kubeadm,在生产环境部署K8S高可用集群(一):https://developer.aliyun.com/article/1495644

Run ‘kubectl get nodes’ to see this node join the cluster.

node节点加入:

kubeadm join 10.10.0.10:7443 --token abcdef.0123456789abcdef

–discovery-token-ca-cert-hash sha256:eac394f46b758da8502c3e25882584432f195c809d29c6038f0fcefc201c8fac

[root@node01 ~ ]# kubeadm join 10.10.0.10:7443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:eac394f46b758da8502c3e25882584432f195c809d29c6038f0fcefc201c8fac

[preflight] Running pre-flight checks

[preflight] Reading configuration from the cluster…

[preflight] FYI: You can look at this config file with ‘kubectl -n kube-system get cm kubeadm-config -o yaml’

[kubelet-start] Writing kubelet configuration to file “/var/lib/kubelet/config.yaml”

[kubelet-start] Writing kubelet environment file with flags to file “/var/lib/kubelet/kubeadm-flags.env”

[kubelet-start] Starting the kubelet

[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap…

This node has joined the cluster:

Certificate signing request was sent to apiserver and a response was received.

The Kubelet was informed of the new secure connection details.

Run ‘kubectl get nodes’ on the control-plane to see this node join the cluster.

在master节点查看集群:

[root@master02 ~ ]# kubectl get nodes

NAME STATUS ROLES AGE VERSION

master01 NotReady control-plane,master 60m v1.21.14

master02 NotReady control-plane,master 25m v1.21.14

master03 NotReady control-plane,master 7m40s v1.21.14

node01 NotReady 2m49s v1.21.14

node02 NotReady 33s v1.21.14

给node节点角色命名:

#可以看到node01、node02的ROLES角色为空,就表示这个节点是工作节点。

#可以把node01和node02的ROLES变成work,按照如下方法:

[root@master01 ~ ]# kubectl label node node01 node-role.kubernetes.io/worker=true

node/node01 labeled

[root@master01 ~ ]# kubectl label node node02 node-role.kubernetes.io/worker=true

node/node02 labeled

[root@master01 ~ ]# kubectl get nodes

NAME STATUS ROLES AGE VERSION

master01 Ready control-plane,master 5h40m v1.21.14

node01 Ready worker 5h31m v1.21.14

node02 Ready worker 159m v1.21.14

[root@master01 ~ ]# kubectl get pod -n kube-system

NAME READY STATUS RESTARTS AGE

coredns-59d64cd4d4-2b6tt 0/1 Pending 0 92m

coredns-59d64cd4d4-7jlws 0/1 Pending 0 92m

etcd-master01 1/1 Running 0 92m

etcd-master02 1/1 Running 0 57m

etcd-master03 1/1 Running 0 38m

kube-apiserver-master01 1/1 Running 0 92m

kube-apiserver-master02 1/1 Running 0 57m

kube-apiserver-master03 1/1 Running 0 38m

kube-controller-manager-master01 1/1 Running 1 92m

kube-controller-manager-master02 1/1 Running 0 57m

kube-controller-manager-master03 1/1 Running 0 38m

kube-proxy-69dmp 1/1 Running 0 92m

kube-proxy-pswl9 1/1 Running 0 57m

kube-proxy-t567z 1/1 Running 0 34m

kube-proxy-wgh9t 1/1 Running 0 38m

kube-proxy-z4mk9 1/1 Running 0 31m

kube-scheduler-master01 1/1 Running 1 92m

kube-scheduler-master02 1/1 Running 0 57m

kube-scheduler-master03 1/1 Running 0 38m

tail -fn300 /var/log/messages

Jul 8 15:55:25 master03 kubelet: I0708 15:55:25.680936 22892 cni.go:239] “Unable to update cni config” err=“no networks found in /etc/cni/net.d”

Jul 8 15:55:26 master03 kubelet: E0708 15:55:26.743448 22892 kubelet.go:2211] “Container runtime network not ready” networkReady=“NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized”

是因为没装网络插件

5.Calico安装

Calico:https://www.projectcalico.org/

https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises

curl https://docs.projectcalico.org/manifests/calico.yaml -O 原版本

点击Manifest下载:

curl https://projectcalico.docs.tigera.io/manifests/calico.yaml -O 新版本

点击requirements 可以查看calico支持的kubernetes的版本

修改calico.yaml的以下位置,把注释取消,改成自己配置的podSubnet网段

- name: CALICO_IPV4POOL_CIDR

value: “172.16.0.0/12”

kubectl apply -f calico.yaml

[root@master01 ~ ]# kubectl apply -f calico.yaml

configmap/calico-config created

customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created

customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created

clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created

clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created

clusterrole.rbac.authorization.k8s.io/calico-node created

clusterrolebinding.rbac.authorization.k8s.io/calico-node created

daemonset.apps/calico-node created

serviceaccount/calico-node created

deployment.apps/calico-kube-controllers created

serviceaccount/calico-kube-controllers created

poddisruptionbudget.policy/calico-kube-controllers created

查看token秘钥:

[root@master01 ~ ]# kubectl get secret -n kube-system

NAME TYPE DATA AGE

attachdetach-controller-token-4s7jv kubernetes.io/service-account-token 3 3h53m

bootstrap-signer-token-7tpd5 kubernetes.io/service-account-token 3 3h53m

bootstrap-token-abcdef bootstrap.kubernetes.io/token 6 3h53m 这个

calico-kube-controllers-token-rhbnf kubernetes.io/service-account-token 3 37m

[root@master01 ~ ]# kubectl get secret -n kube-system bootstrap-token-abcdef -oyaml

apiVersion: v1

data:

auth-extra-groups: c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=

expiration: MjAyMi0wNy0wOVQxMzoxMDozMCswODowMA== token过期时间,是通过base64加密的,可以解密

token-id: YWJjZGVm

token-secret: MDEyMzQ1Njc4OWFiY2RlZg==

usage-bootstrap-authentication: dHJ1ZQ==

usage-bootstrap-signing: dHJ1ZQ==

kind: Secret

metadata:

creationTimestamp: “2022-07-08T05:10:30Z”

name: bootstrap-token-abcdef

namespace: kube-system

resourceVersion: “372”

uid: 83b63111-373a-471b-9530-bfbe55763326

type: bootstrap.kubernetes.io/token

[root@master01 ~ ]# echo “MjAyMi0wNy0wOVQxMzoxMDozMCswODowMA==”|base64 -d

2022-07-09T13:10:30+08:00[root@master01 ~ ]#

token一天就过期,过期后可以使用kubeadm来重新生成token

假如这个token过期,我们先把它删掉:

[root@master01 ~ ]# kubectl delete secret -n kube-system bootstrap-token-abcdef

secret “bootstrap-token-abcdef” deleted

重新生成node节点加入的token:

[root@master01 ~ ]# kubeadm token create --print-join-command

kubeadm join 10.10.0.10:7443 --token hbfk39.j5aj5rcejq401rhb --discovery-token-ca-cert-hash sha256:eac394f46b758da8502c3e25882584432f195c809d29c6038f0fcefc201c8fac

查看过期时间:

[root@master01 ~ ]# kubectl get secret bootstrap-token-hbfk39 -n kube-system -oyaml

apiVersion: v1

data:

auth-extra-groups: c3lzdGVtOmJvb3RzdHJhcHBlcnM6a3ViZWFkbTpkZWZhdWx0LW5vZGUtdG9rZW4=

expiration: MjAyMi0wNy0wOVQxNzoyMDo0NiswODowMA==

token-id: aGJmazM5

token-secret: ajVhajVyY2VqcTQwMXJoYg==

usage-bootstrap-authentication: dHJ1ZQ==

usage-bootstrap-signing: dHJ1ZQ==

kind: Secret

metadata:

creationTimestamp: “2022-07-08T09:20:48Z”

name: bootstrap-token-hbfk39

namespace: kube-system

resourceVersion: “23993”

uid: 2904221d-7da7-4656-8fc8-30c32cade8d1

type: bootstrap.kubernetes.io/token

生成master的 --certificate-key:

[root@master01 ~ ]# kubeadm init phase upload-certs --upload-certs

I0708 17:29:52.123627 105799 version.go:254] remote version is much newer: v1.24.2; falling back to: stable-1.21

[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace

[upload-certs] Using certificate key:

35652b0c344704df60bdab4e4d2386b93d12f31d719263cddccbbcf32bb8764c

其他master加入只需要修改 --certificate-key为这个key即可

kubeadm join 10.10.0.10:7443 --token abcdef.0123456789abcdef

–discovery-token-ca-cert-hash sha256:35652b0c344704df60bdab4e4d2386b93d12f31d719263cddccbbcf32bb8764c --control-plane --certificate-key 35652b0c344704df60bdab4e4d2386b93d12f31d719263cddccbbcf32bb8764c

6.Metrics部署

在新版的Kubernetes中系统资源的采集均使用Metrics-server,可以通过Metrics采集节点和Pod的内存、磁盘、CPU和网络的使用率。

以前比较老的版本使用Heapster,现在已经被弃用

下载软件包:

git clone https://github.com/dotbalo/k8s-ha-install.git

安装metrics-server

·官网:https://github.com/kubernets-sigs/metrics-server

https://github.com/kubernetes-sigs/metrics-server

下载镜像和资源文件,并导入私有仓库

镜像导入

安装metrics-server

——rbac.yaml 授权控制器

——pdb.yaml 中断控制器

——development.yaml 主进程 metrics

——service.yaml 后端是 metrics 主进程的服务

——apiservice.yaml 注册集群API

下载软件包:

git clone https://github.com/dotbalo/k8s-ha-install.git

[root@master01 metrics-server-3.6.1 ]# pwd

/root/k8s-ha-install/metrics-server-3.6.1

安装:

[root@master01 metrics-server-3.6.1 ]# kubectl apply -f .

clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created

clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created

rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created

Warning: apiregistration.k8s.io/v1beta1 APIService is deprecated in v1.19+, unavailable in v1.22+; use apiregistration.k8s.io/v1 APIService

apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

serviceaccount/metrics-server created

deployment.apps/metrics-server created

service/metrics-server created

clusterrole.rbac.authorization.k8s.io/system:metrics-server created

clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created

注:如果失败,删除metrics-server资源:

kubectl delete -f metrics-server.yaml

github官网下载安装metrics-server:

https://github.com/kubernetes-sigs/metrics-server/releases

https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.6.0/components.yaml

先修改两个地方:

containers:

- args:

- --cert-dir=/tmp

- --secure-port=4443

- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname

- --kubelet-use-node-status-port

- --metric-resolution=15s

- --kubelet-insecure-tls # 加上该启动参数

image: registry.cn-hangzhou.aliyuncs.com/zailushang/metrics-server:v0.6.0 # 国内集群,请替换成这个镜像

[root@master01 ~ ]# kubectl apply -f components.yaml

serviceaccount/metrics-server created

clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created

clusterrole.rbac.authorization.k8s.io/system:metrics-server created

rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created

clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created

clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created

service/metrics-server created

deployment.apps/metrics-server created

apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created

查看状态:

[root@master01 ~ ]# kubectl get pod -n kube-system -l k8s-app=metrics-server

NAME READY STATUS RESTARTS AGE

metrics-server-7f44b488b9-gljj2 1/1 Running 0 11m

kubectl top命令可显示节点和Pod对象的资源使用信息,它依赖于集群中的资源指标API来收集各项指标数据。

它包含有node和pod两个命令,可分别用于显示Node对象和Pod对象的相关资源占用率。

[root@master01 ~ ]# kubectl top nodes --use-protocol-buffers

NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%

master01 282m 14% 1161Mi 62%

master02 333m 16% 1256Mi 67%

master03 296m 14% 1237Mi 66%

node01 134m 3% 1044Mi 55%

1、查看集群资源占用

[root@master01 ~ ]# kubectl top pod --use-protocol-buffers -A

NAMESPACE NAME CPU(cores) MEMORY(bytes)

kube-system calico-kube-controllers-69f595f8f8-9tgr4 2m 34Mi

kube-system calico-node-7htlt 31m 135Mi

kube-system calico-node-8j7zg 55m 80Mi

kube-system calico-node-rgwsf 32m 135Mi

kube-system calico-node-tgwjx 36m 143Mi

kube-system coredns-59d64cd4d4-2b6tt 3m 25Mi

kube-system coredns-59d64cd4d4-7jlws 5m 17Mi

kube-system etcd-master01 43m 89Mi

kube-system etcd-master02 59m 90Mi

kube-system etcd-master03 47m 88Mi

kube-system kube-apiserver-master01 72m 331Mi

kube-system kube-apiserver-master02 82m 384Mi

kube-system kube-apiserver-master03 88m 366Mi

kube-system kube-controller-manager-master01 2m 32Mi

kube-system kube-controller-manager-master02 20m 90Mi

kube-system kube-controller-manager-master03 3m 39Mi

kube-system kube-proxy-69dmp 1m 34Mi

kube-system kube-proxy-pswl9 1m 32Mi

kube-system kube-proxy-wgh9t 1m 30Mi

kube-system kube-proxy-wj5nm 1m 18Mi

kube-system kube-scheduler-master01 6m 30Mi

kube-system kube-scheduler-master02 4m 41Mi

kube-system kube-scheduler-master03 3m 35Mi

kube-system metrics-server-7f44b488b9-gljj2 5m 25Mi

kubernetes-dashboard dashboard-metrics-scraper-7c857855d9-rqwm5 1m 14Mi

kubernetes-dashboard kubernetes-dashboard-bcf9d8968-5wh7b 1m 25Mi

查看内存较大的pod,基于内存占用排序:

[root@k8s-master shell ]#kubectl top pod --use-protocol-buffers -A|sed 1d|awk -F “M” ‘{print $1}’|sort -k4nr

kube-logging es-cluster-0 91m 1148

kube-logging es-cluster-1 99m 1093

tools tools-7c45b75fd7-chxrx 1m 927

kube-logging es-cluster-2 86m 922

storage tfies-0 2m 887

monitor-sa prometheus-server-7474845b45-tvlck 38m 882

storage tfies-1 2m 822

cattle-prometheus prometheus-cluster-monitoring-0 21m 678

gateway gateway-69bfb5df5c-gm8zv 7m 559

查看节点详细信息:

kubectl describe node master01

7.Dashboard部署

官方GitHub:https://github.com/kubernetes/dashboard

在谷歌浏览器(Chrome)启动文件中加入启动参数,用于解决无法访问Dashboard的问题,参考图1-1:谷歌浏览器快捷键属性,目标文件的最后添加

–test-type --ignore-certificate-errors

安装最新版:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.0/aio/deploy/recommended.yaml

[root@master01 ~ ]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.0/aio/deploy/recommended.yaml

namespace/kubernetes-dashboard created

serviceaccount/kubernetes-dashboard created

service/kubernetes-dashboard created

secret/kubernetes-dashboard-certs created

secret/kubernetes-dashboard-csrf created

secret/kubernetes-dashboard-key-holder created

configmap/kubernetes-dashboard-settings created

role.rbac.authorization.k8s.io/kubernetes-dashboard created

clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created

rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

deployment.apps/kubernetes-dashboard created

service/dashboard-metrics-scraper created

deployment.apps/dashboard-metrics-scraper created

如果不能安装,需要将文件打开复制出来编辑个yaml文件

查看dashboard:

[root@master01 ~ ]# kubectl get po -n kubernetes-dashboard

NAME READY STATUS RESTARTS AGE

dashboard-metrics-scraper-7c857855d9-tqhjk 1/1 Running 0 4m6s

kubernetes-dashboard-bcf9d8968-b9td5 1/1 Running 0 4m6s

更改dashboard的svc为NodePort:在每个宿主机上启动一个端口

kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

[root@master01 ~ ]# kubectl get svc kubernetes-dashboard -n kubernetes-dashboard

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

kubernetes-dashboard NodePort 192.168.20.101 443:31459/TCP 2d20h

根据自己的实例端口号,通过任意安装了kube-proxy的宿主机或者VIP的IP+端口即可访问到dashboard:

https://10.10.0.224:31459

查看有没有管理员用户:

[root@master01 ~ ]# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk ‘{print $1}’)

创建管理员用户vim admin.yaml

[root@master01 ~ ]# vim admin.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding 
metadata: 
  name: admin-user
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system


查看token值:

[root@master01 ~ ]# kubectl apply -f admin.yaml -n kube-system
serviceaccount/admin-user created
clusterrolebinding.rbac.authorization.k8s.io/admin-user created


[root@master01 ~ ]# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
Name:         admin-user-token-6dpml
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: admin-user
              kubernetes.io/service-account.uid: a871d4cf-28a4-4d36-b3a7-917efbbace02

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1066 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IlVFMHY0TjJuY3BzRmVPejRtZUFNSlp3TWs4bExNYjhpZWR0dVNDdEt3ZjgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLTZkcG1sIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhODcxZDRjZi0yOGE0LTRkMzYtYjNhNy05MTdlZmJiYWNlMDIiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.zdtLkxuyB3LpWbx7G24cbRMjlcS9LO1xnIkQrhUcPutqH_IDNMOD7a5KL-5QLVZoJyyf5_w6oAfsdK24fXcKspSrNiFbl_N79BP8Ktqur8NbP06giU3bFH4rhchjNaGJNkpkpl9j99_8tSIUPvH_BCQsLryynLwi9S7bsojEwG-iMLvtUbfuwudEaQv4oN-OfNcFNlqlSAMTAto65WtLhTn4YV6XAjSAFebrbhpnUs06_JRUdnpZooSuhkCquAW7cuKJbCTTkVly5jBuvdQA67cQsbb9V82k2S97NK6ov5WhXH2nY1GrMDPa5xLJD_kkUvOvVZXCfF2YelAwtr3oZQ

https://10.10.0.224:31459

选择token,输入token值即可登录

查看所有集群容器:

[root@master01 ~ ]# kubectl get pod --all-namespaces

集群验证:

kubernetes ip:service网段的第一个地址

[root@master01 ~ ]# kubectl get svc

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

kubernetes ClusterIP 192.168.0.1 443/TCP 2d21h

DNS service网段第十个地址:

[root@master01 ~ ]# kubectl get svc -n kube-system

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

kube-dns ClusterIP 192.168.0.10 53/UDP,53/TCP,9153/TCP 2d21h

metrics-server ClusterIP 192.168.233.65 443/TCP 2d16h

[root@master01 ~ ]# kubectl get pod --all-namespaces -owide

NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES

kube-system calico-kube-controllers-69f595f8f8-9tgr4 1/1 Running 7 2d18h 172.20.59.200 master02

kube-system calico-node-7htlt 1/1 Running 2 2d18h 10.10.0.221 master02

kube-system calico-node-8j6gc 1/1 Running 2 2d18h 10.10.0.224 node01

kube-system calico-node-rgwsf 1/1 Running 5 2d18h 10.10.0.220 master01

kube-system calico-node-sxczq 0/1 Running 0 2d18h 10.10.0.225 node02

kube-system calico-node-tgwjx 1/1 Running 2 2d18h 10.10.0.223 master03

kube-system coredns-59d64cd4d4-2b6tt 1/1 Running 2 2d22h 172.20.59.199 master02

kube-system coredns-59d64cd4d4-7jlws 1/1 Running 2 2d22h 172.20.59.201 master02

kube-system etcd-master01 1/1 Running 9 2d22h 10.10.0.220 master01

kube-system etcd-master02 1/1 Running 6 2d21h 10.10.0.221 master02

kube-system etcd-master03 1/1 Running 6 2d21h 10.10.0.223 master03

kube-system kube-apiserver-master01 1/1 Running 12 2d22h 10.10.0.220 master01

kube-system kube-apiserver-master02 1/1 Running 10 2d21h 10.10.0.221 master02

kube-system kube-apiserver-master03 1/1 Running 14 2d21h 10.10.0.223 master03

kube-system kube-controller-manager-master01 1/1 Running 11 2d22h 10.10.0.220 master01

kube-system kube-controller-manager-master02 1/1 Running 6 2d21h 10.10.0.221 master02

kube-system kube-controller-manager-master03 1/1 Running 4 2d21h 10.10.0.223 master03

kube-system kube-proxy-69dmp 1/1 Running 4 2d22h 10.10.0.220 master01

kube-system kube-proxy-pswl9 1/1 Running 2 2d21h 10.10.0.221 master02

kube-system kube-proxy-t567z 1/1 Running 2 2d21h 10.10.0.224 node01

kube-system kube-proxy-wgh9t 1/1 Running 2 2d21h 10.10.0.223 master03

kube-system kube-proxy-z4mk9 1/1 Running 1 2d21h 10.10.0.225 node02

kube-system kube-scheduler-master01 1/1 Running 11 2d22h 10.10.0.220 master01

kube-system kube-scheduler-master02 1/1 Running 5 2d21h 10.10.0.221 master02

kube-system kube-scheduler-master03 1/1 Running 5 2d21h 10.10.0.223 master03

kube-system metrics-server-769bd9c6f4-hvtlq 0/1 CrashLoopBackOff 41 2d17h 172.29.55.9 node01

kubernetes-dashboard dashboard-metrics-scraper-7c857855d9-tqhjk 1/1 Running 2 2d17h 172.29.55.7 node01

kubernetes-dashboard kubernetes-dashboard-bcf9d8968-b9td5 1/1 Running 3 2d17h 172.29.55.8 node01

进入到容器中执行命令:

[root@master01 ~ ]# kubectl exec -it calico-node-8j6gc -n kube-system – sh

Defaulted container “calico-node” out of: calico-node, upgrade-ipam (init), install-cni (init)

sh-4.4#

查看日志:

[root@master01 ~ ]# kubectl logs -n kube-system metrics-server-769bd9c6f4-hvtlq

删除pod:

[root@master01 metrics-server-3.6.1 ]# kubectl delete pod metrics-server-769bd9c6f4-hvtlq -n kube-system

pod “metrics-server-769bd9c6f4-hvtlq” deleted

将Kube-proxy改为ipvs模式,因为在初始化集群的时候注释了ipvs配置,所以需要自行修改一下:

2.11 一些必须的配置更改

将Kube-proxy改为ipvs模式,因为在初始化集群的时候注释了ipvs配置,所以需要自行修改一下:

在master01节点执行

kubectl edit cm kube-proxy -n kube-system

mode: “ipvs”

注意事项

注意:kubeadm安装的集群,证书有效期默认是一年。

master节点的kube-apiserver、kube-scheduler、kube-controller-manager、etcd都是以容器运行的。

可以通过kubectl get po -n kube-system查看。

启动和二进制不同的是,kubelet的配置文件在/etc/sysconfig/Kubelet

其他组件的配置文件在/etc/kubernetes目录下,比如kube-apiserver.yaml,该yaml文件更改后,

kubelet会自动刷新配置,也就是会重启pod。不能再次创建该文件

dashboard使用过程中很不方便,浏览器里面的编辑,添加等操作都是yaml格式形式编辑的

国内开发的https://kuboard.cn/ 比较好用

https://kuboard.cn/install/v3/install-in-k8s.html#%E6%96%B9%E6%B3%95%E4%B8%80-%E4%BD%BF%E7%94%A8-hostpath-%E6%8F%90%E4%BE%9B%E6%8C%81%E4%B9%85%E5%8C%96

选择在线安装:

kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml

查看:

[root@master01 ~ ]# kubectl get nodes -n kuboard

NAME STATUS ROLES AGE VERSION

master01 Ready control-plane,master 3d3h v1.21.14

master02 Ready control-plane,master 3d3h v1.21.14

master03 Ready control-plane,master 3d2h v1.21.14

node01 Ready 167m v1.21.14

node02 NotReady 3d2h v1.21.14

查看服务:

[root@master01 ~ ]# kubectl get svc -n kuboard

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

kuboard-v3 NodePort 192.168.16.221 80:30080/TCP,10081:30081/TCP,10081:30081/UDP 23m

访问 Kuboard

在浏览器中打开链接 http://your-node-ip-address:30080

输入初始用户名和密码,并登录

用户名: admin

密码: Kuboard123

在浏览器导入kuboard-agent

删除节点:

[root@master01 cfg ]# kubectl delete node master01

相关实践学习
容器服务Serverless版ACK Serverless 快速入门:在线魔方应用部署和监控
通过本实验,您将了解到容器服务Serverless版ACK Serverless 的基本产品能力,即可以实现快速部署一个在线魔方应用,并借助阿里云容器服务成熟的产品生态,实现在线应用的企业级监控,提升应用稳定性。
云原生实践公开课
课程大纲 开篇:如何学习并实践云原生技术 基础篇: 5 步上手 Kubernetes 进阶篇:生产环境下的 K8s 实践 相关的阿里云产品:容器服务&nbsp;ACK 容器服务&nbsp;Kubernetes&nbsp;版(简称&nbsp;ACK)提供高性能可伸缩的容器应用管理能力,支持企业级容器化应用的全生命周期管理。整合阿里云虚拟化、存储、网络和安全能力,打造云端最佳容器化应用运行环境。 了解产品详情:&nbsp;https://www.aliyun.com/product/kubernetes
相关文章
|
6天前
|
Kubernetes Cloud Native 微服务
微服务实践之使用 kube-vip 搭建高可用 Kubernetes 集群
微服务实践之使用 kube-vip 搭建高可用 Kubernetes 集群
181 3
|
21天前
|
Kubernetes 开发工具 Docker
微服务实践k8s与dapr开发部署实验(2)状态管理
微服务实践k8s与dapr开发部署实验(2)状态管理
52 3
微服务实践k8s与dapr开发部署实验(2)状态管理
|
8天前
|
Kubernetes 数据处理 调度
天呐!部署 Kubernetes 模式的 Havenask 集群太震撼了!
【6月更文挑战第11天】Kubernetes 与 Havenask 集群结合,打造高效智能的数据处理解决方案。Kubernetes 如指挥家精准调度资源,Havenask 快速响应查询,简化复杂任务,优化资源管理。通过搭建 Kubernetes 环境并配置 Havenask,实现高可扩展性和容错性,保障服务连续性。开发者因此能专注业务逻辑,享受自动化基础设施管理带来的便利。这项创新技术组合引领未来,开启数据处理新篇章。拥抱技术新时代!
|
15天前
|
Kubernetes 容器 Perl
k8s部署seata 报错 没有提供足够的身份验证信息 [ http-nio-7091-exec-2] [ty.JwtAuthenticationEntryPoint] [ commence] [] : Responding with unauthorized error. Message - Full authentication is required to access this resource
Kubernetes pod 在16:12时出现两次错误,错误信息显示需要完整认证才能访问资源。尽管有此错误,但页面可正常访问。附有yaml配置文件的图片。
31 2
|
18天前
|
Kubernetes 数据安全/隐私保护 Docker
kubeadm 工具实验 k8s一键安装
kubeadm 工具实验 k8s一键安装
|
18天前
|
Kubernetes 负载均衡 应用服务中间件
k8s 二进制安装 优化架构之 部署负载均衡,加入master02
k8s 二进制安装 优化架构之 部署负载均衡,加入master02
|
Kubernetes 开发者 微服务
简化Kubernetes应用部署工具-Helm之Hook
微服务和容器化给复杂应用部署与管理带来了极大的挑战。Helm是目前Kubernetes服务编排领域的唯一开源子项目,做为Kubernetes应用的一个包管理工具,可理解为Kubernetes的apt-get / yum,由Deis 公司发起,该公司已经被微软收购。
1577 0
|
Kubernetes 开发者 微服务
简化Kubernetes应用部署工具-Helm之Hook
本文讲的是简化Kubernetes应用部署工具-Helm之Hook【编者的话】微服务和容器化给复杂应用部署与管理带来了极大的挑战。Helm是目前Kubernetes服务编排领域的唯一开源子项目,做为Kubernetes应用的一个包管理工具,可理解为Kubernetes的apt-get / yum,由Deis 公司发起,该公司已经被微软收购。
2529 0
|
17天前
|
Kubernetes 微服务 容器
Aspire项目发布到远程k8s集群
Aspire项目发布到远程k8s集群
360 2
Aspire项目发布到远程k8s集群
|
8天前
|
Kubernetes 前端开发 Serverless
Serverless 应用引擎产品使用合集之如何调用Kubernetes集群内服务
阿里云Serverless 应用引擎(SAE)提供了完整的微服务应用生命周期管理能力,包括应用部署、服务治理、开发运维、资源管理等功能,并通过扩展功能支持多环境管理、API Gateway、事件驱动等高级应用场景,帮助企业快速构建、部署、运维和扩展微服务架构,实现Serverless化的应用部署与运维模式。以下是对SAE产品使用合集的概述,包括应用管理、服务治理、开发运维、资源管理等方面。