metrics-server用于监测node,pod等的CPU，内存使用情况（hpa弹性伸缩依赖metrics-server插件）
1.13.0、创建metrics-server证书和私钥
k8s-01:~ # cd /opt/k8s/ssl/ k8s-01:/opt/k8s/ssl # source /opt/k8s/bin/k8s-env.sh k8s-01:/opt/k8s/ssl # cat > metrics-server-csr.json <<EOF { "CN": "aggregator", "hosts": [ ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "ShangHai", "L": "ShangHai", "O": "k8s", "OU": "bandian" } ] } EOF
1.13.1、生成metrics-server证书和私钥
k8s-01:/opt/k8s/ssl # cfssl gencert -ca=/opt/k8s/ssl/ca.pem \ -ca-key=/opt/k8s/ssl/ca-key.pem \ -config=/opt/k8s/ssl/ca-config.json \ -profile=kubernetes metrics-server-csr.json | cfssljson -bare metrics-server
1.13.2、开启kube-apiserver聚合配置
- 在kube-apiserver.service文件里面，增加如下内容，用来开启聚合（此操作，后面需要重启kube-apiserver组件，建议在部署kube-apiserver的时候，就开启聚合）
--proxy-client-cert-file=/etc/kubernetes/cert/metrics-server.pem \\ --proxy-client-key-file=/etc/kubernetes/cert/metrics-server-key.pem \\ --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\ --requestheader-allowed-names=aggregator \\ --requestheader-extra-headers-prefix="X-Remote-Extra-" \\ --requestheader-group-headers=X-Remote-Group \\ --requestheader-username-headers=X-Remote-User
- 为了方便，就重新创建kube-apiserver.service文件了（注意自己kube-apiserver的service文件，别直接复制黏贴我的配置文件）
k8s-01:~ # cd /opt/k8s/conf/ k8s-01:/opt/k8s/conf # cat > kube-apiserver.service.template <<EOF [Unit] Description=Kubernetes API Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=network.target [Service] WorkingDirectory=${K8S_DIR}/kube-apiserver ExecStart=/opt/k8s/bin/kube-apiserver \\ --v=2 \\ --advertise-address=##NODE_IP## \\ --secure-port=6443 \\ --bind-address=##NODE_IP## \\ --etcd-servers=${ETCD_ENDPOINTS} \\ --allow-privileged=true \\ --service-cluster-ip-range=${SERVICE_CIDR} \\ --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\ --authorization-mode=RBAC,Node \\ --enable-bootstrap-token-auth=true \\ --token-auth-file=/etc/kubernetes/cert/token.csv \\ --service-node-port-range=${NODE_PORT_RANGE} \\ --kubelet-client-certificate=/etc/kubernetes/cert/kubernetes.pem \\ --kubelet-client-key=/etc/kubernetes/cert/kubernetes-key.pem \\ --tls-cert-file=/etc/kubernetes/cert/kubernetes.pem \\ --tls-private-key-file=/etc/kubernetes/cert/kubernetes-key.pem \\ --client-ca-file=/etc/kubernetes/cert/ca.pem \\ --service-account-key-file=/etc/kubernetes/cert/ca.pem \\ --etcd-cafile=/etc/kubernetes/cert/ca.pem \\ --etcd-certfile=/etc/kubernetes/cert/kubernetes.pem \\ --etcd-keyfile=/etc/kubernetes/cert/kubernetes-key.pem \\ --audit-log-maxage=15 \\ --audit-log-maxbackup=3 \\ --audit-log-maxsize=100 \\ --audit-log-truncate-enabled \\ --audit-log-path=${K8S_DIR}/kube-apiserver/audit.log \\ --proxy-client-cert-file=/etc/kubernetes/cert/metrics-server.pem \\ --proxy-client-key-file=/etc/kubernetes/cert/metrics-server-key.pem \\ --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\ --requestheader-allowed-names=aggregator \\ --requestheader-extra-headers-prefix="X-Remote-Extra-" \\ --requestheader-group-headers=X-Remote-Group \\ --requestheader-username-headers=X-Remote-User Restart=on-failure RestartSec=10 Type=notify LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF
1.13.3、分发配置文件和秘钥到其他节点
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh # 替换模板文件 for (( i=0; i < 3; i++ )) do sed -e "s/##NODE_IP##/${MASTER_IPS[i]}/" /opt/k8s/conf/kube-apiserver.service.template > \ /opt/k8s/conf/kube-apiserver-${MASTER_IPS[i]}.service done for host in ${MASTER_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" scp /opt/k8s/ssl/metrics-server*.pem ${host}:/etc/kubernetes/cert/ scp /opt/k8s/conf/kube-apiserver-${host}.service ${host}:/etc/systemd/system/kube-apiserver.service done
1.13.4、重启所有的kube-apiserver组件
#!/usr/bin/env bash source /opt/k8s/bin/k8s-env.sh for host in ${MASTER_IPS[@]} do printf "\e[1;34m${host}\e[0m\n" ssh root@${host} "systemctl daemon-reload && \ systemctl restart kube-apiserver && \ systemctl status kube-apiserver | grep Active" done
1.13.5、下载yaml文件
k8s-01:~ # wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
1.13.6、配置yaml文件
- 由于github上面拉取的yaml当中，有许多内容需要修改，因此，下面将修改好的yaml文件放上来了，可以直接使用
k8s-01:~ # vim components.yaml
--- --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:aggregated-metrics-reader labels: rbac.authorization.k8s.io/aggregate-to-view: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-admin: "true" rules: - apiGroups: ["metrics.k8s.io"] resources: ["pods", "nodes"] verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: name: v1beta1.metrics.k8s.io spec: service: name: metrics-server namespace: kube-system group: metrics.k8s.io version: v1beta1 insecureSkipTLSVerify: true groupPriorityMinimum: 100 versionPriority: 100 --- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 imagePullPolicy: IfNotPresent args: - --cert-dir=/tmp - --secure-port=4443 - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP ports: - name: main-port containerPort: 4443 protocol: TCP securityContext: readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 volumeMounts: - name: tmp-dir mountPath: /tmp nodeSelector: kubernetes.io/os: linux kubernetes.io/arch: "amd64" --- apiVersion: v1 kind: Service metadata: name: metrics-server namespace: kube-system labels: kubernetes.io/name: "Metrics-server" kubernetes.io/cluster-service: "true" spec: selector: k8s-app: metrics-server ports: - port: 443 protocol: TCP targetPort: main-port --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - nodes/stats - namespaces - configmaps verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system
k8s-01:~ # kubectl apply -f components.yaml
1.13.7、验证metrics-server功能
- metrics-server启动会比较慢，耐心
等待1-3分钟，出现如下效果，则成功 - 若没有出现，则使用
kubectl logs -n kube-system metrics-server-xxxx查看日志
k8s-01:~ # kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% 192.168.72.55 129m 6% 2232Mi 58% 192.168.72.56 119m 5% 1555Mi 40% 192.168.72.57 114m 5% 1425Mi 37% 192.168.72.58 31m 1% 711Mi 18% 192.168.72.59 28m 1% 733Mi 19% k8s-01:~ # kubectl top pod -A NAMESPACE NAME CPU(cores) MEMORY(bytes) kube-system coredns-689d7d9f49-s2qjn 2m 13Mi kube-system coredns-689d7d9f49-vc9k4 3m 17Mi kube-system metrics-server-666566b66d-jfl7v 2m 12Mi
