下面有metrics-server的完整yaml文件内容

一、HPA概述

HPA全称Horizontal Pod Autoscaler，即水平Pod自动伸缩器，可以根据观察到的CPU、内存使用率或自定义度量标准来自动增加或者减少Pod的数量，但是HPA不适用于无法扩、缩容的对象，例如DaemonSet，通常都作用与Deployment
HPA控制器会定期调整RC或者Deployment的副本数，使对象数量符合用户定义规则的数量
既然是通过CPU、内存等指标来自动扩、缩容Pod，那么HPA肯定是需要一个能监控这些硬件资源的组件，则例的组件有很多选择，例如metrices-server、Heapster等，这里使用metrices-server

metrices-server从api-server中获取cpu、内存使用率等监控指标

二、HPA版本

查看HPA所有版本

[root@master C]# kubectl api-versions |grep autoscaling
autoscaling/v1      #只支持通过CPU为参考依据来改变Pod的副本数
autoscaling/v2beta1   #支持通过CPU、内存、连接数或者自定义规则为参考依据
autoscaling/v2beta2   #和v2beta1差不多

查看当前版本

[root@master C]# kubectl explain hpa
KIND:     HorizontalPodAutoscaler
VERSION:  autoscaling/v1  #可以看到使用的默认版本是v1
DESCRIPTION:
     configuration of a horizontal pod autoscaler.
FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
   metadata     <Object>
     Standard object metadata. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
   spec <Object>
     behaviour of autoscaler. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.
   status       <Object>
     current information about the autoscaler.

指定使用版本，这里并不是修改，相当于执行这条命令时，指定了下版本

[root@master C]# kubectl explain hpa --api-version=autoscaling/v2beta1
KIND:     HorizontalPodAutoscaler
VERSION:  autoscaling/v2beta1
DESCRIPTION:
     HorizontalPodAutoscaler is the configuration for a horizontal pod
     autoscaler, which automatically manages the replica count of any resource
     implementing the scale subresource based on the metrics specified.
FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
   metadata     <Object>
     metadata is the standard object metadata. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
   spec <Object>
     spec is the specification for the behaviour of the autoscaler. More info:
     https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.
   status       <Object>
     status is the current information about the autoscaler.

三、HPA部署

（1）部署metrics-server

[root@master kube-system]# kubectl top nodes  #查看节点状态，因为没有安装，所以会报错
Error from server (NotFound): the server could not find the requested resource (get services http:heapster:)

编写yaml文件，注意端口和镜像

[root@master kube-system]# vim components-v0.5.0.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls
        image: registry.cn-shenzhen.aliyuncs.com/zengfengjin/metrics-server:v0.5.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          initialDelaySeconds: 20
          periodSeconds: 10
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

部署

[root@master kube-system]# kubectl apply -f components-v0.5.0.yaml
serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
#查看创建的pod
[root@master kube-system]# kubectl get pods -n kube-system| egrep 'NAME|metrics-server'
NAME                              READY   STATUS              RESTARTS   AGE
metrics-server-5944675dfb-q6cdd   0/1     ContainerCreating   0          6s
 #查看日志
[root@master kube-system]# kubectl logs metrics-server-5944675dfb-q6cdd  -n kube-system 
I0718 03:06:39.064633       1 serving.go:341] Generated self-signed cert (/tmp/apiserver.crt, /tmp/apiserver.key)
I0718 03:06:39.870097       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0718 03:06:39.870122       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0718 03:06:39.870159       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0718 03:06:39.870160       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0718 03:06:39.870105       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0718 03:06:39.871166       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
I0718 03:06:39.872804       1 dynamic_serving_content.go:130] Starting serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key
I0718 03:06:39.875741       1 secure_serving.go:197] Serving securely on [::]:4443
I0718 03:06:39.876050       1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0718 03:06:39.970469       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0718 03:06:39.970575       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0718 03:06:39.971610       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController
#如果报错的化，可以修改apiserver的yaml文件，这是k8s的yaml文件
[root@master kube-system]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
 40     - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
 41     - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
 42     - --enable-aggregator-routing=true     #添加这行
 43     image: registry.aliyuncs.com/google_containers/kube-apiserver:v1.18.0
 44     imagePullPolicy: IfNotPresent
#保存退出
[root@master kube-system]# systemctl restart kubelet  #修改后重启kubelet
#再次查看节点信息
[root@master kube-system]# kubectl top node
NAME     CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
master   327m         4%     3909Mi          23%
node     148m         1%     1327Mi          8%

（2）创建Deployment

这里创建一个nginx的deployment

[root@master test]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      run: nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.2
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
          requests:  #想要HPA生效，必须添加requests声明
            cpu: 200m
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    run: nginx
spec:
  ports:
  - port: 80
  selector:
    run: nginx

访问测试

[root@master test]# kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
nginx-9cb8d65b5-tq9v4   1/1     Running   0          14m   10.244.1.22   node   <none>           <none>
[root@master test]# kubectl get svc nginx
NAME    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginx   ClusterIP   172.16.169.27   <none>        80/TCP    15m
[root@master test]# kubectl describe svc nginx
Name:              nginx
Namespace:         default
Labels:            run=nginx
Annotations:       Selector:  run=nginx
Type:              ClusterIP
IP:                172.16.169.27
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.22:80
Session Affinity:  None
Events:            <none>
[root@node test]# curl 172.16.169.27  #访问成功
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

（3）基于CPU创建HPA

#创建一个cpu利用率达到20，最大10个pod，最小1个，这里没有指定版本所以默认是v1版本，而v1版本只能以CPU为标准
[root@master test]# kubectl autoscale deployment nginx --cpu-percent=20 --min=1 --max=10
horizontalpodautoscaler.autoscaling/nginx autoscaled
#TARGETS可以看到使用率
[root@master test]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/20%    1         10        1          86s
#创建一个测试pod增加负载，访问地址要和pod的svc地址相同
[root@master ~]# kubectl  run busybox -it --image=busybox -- /bin/sh -c 'while true; do wget -q -O- http://10.244.1.22; done'
#过一分钟后看hap的使用率，REPLICAS是当前pod的数量
[root@master test]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   27%/20%   1         10        5          54m
[root@master test]# kubectl get pods   #再看pod数量，发现已经增加到了5个
NAME                    READY   STATUS    RESTARTS   AGE
bustbox                 1/1     Running   0          119s
nginx-9cb8d65b5-24dg2   1/1     Running   0          57s
nginx-9cb8d65b5-c6n98   1/1     Running   0          87s
nginx-9cb8d65b5-ksjzv   1/1     Running   0          57s
nginx-9cb8d65b5-n77fm   1/1     Running   0          87s
nginx-9cb8d65b5-tq9v4   1/1     Running   0          84m
[root@master test]# kubectl get deployments.apps
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   5/5     5            5           84m
#此时，停止压测，过好几分钟后再次查看pod数量和使用率
[root@master test]# kubectl delete pod busybox  #终止后，删除pod
[root@master test]# kubectl get hpa  #虽然使用率已经降到0了，但是可以看到当前REPLICAS的数量还5，这个需要等一会就会缩容
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/20%    1         10        5          58m
#过了几分钟后，可以看到pod数量已经回到了1
[root@master test]# kubectl get hpa
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx   Deployment/nginx   0%/20%    1         10        1          64m  
[root@master test]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-9cb8d65b5-tq9v4   1/1     Running   0          95m

（4）基于内存创建的HPA

#先把上面创建的资源删除
[root@master test]# kubectl delete horizontalpodautoscalers.autoscaling  nginx
horizontalpodautoscaler.autoscaling "nginx" deleted
[root@master test]# kubectl delete -f nginx.yaml
deployment.apps "nginx" deleted
service "nginx" deleted

重新编写yaml文件

[root@master test]# cat nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      run: nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.2
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
            memory: 60Mi
          requests:
            cpu: 200m
            memory: 25Mi
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    run: nginx
spec:
  ports:
  - port: 80
  selector:
    run: nginx
[root@master test]# kubectl apply -f nginx.yaml
deployment.apps/nginx created
service/nginx created

创建HPA

[root@master test]# vim hpa-nginx.yaml
apiVersion: autoscaling/v2beta1  #上面的hpa版本有提到过，使用基于内存的hpa需要换个版本
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-hpa
spec:
  maxReplicas: 10  #1-10的pod数量限制
  minReplicas: 1
  scaleTargetRef:               #指定使用hpa的资源对象，版本、类型、名称要和上面创建的相同
    apiVersion: apps/v1
    kind: Deployment
    name: nginx
  metrics:
  - type: Resource
    resource:
      name: memory
      targetAverageUtilization: 50   #限制%50的内存

[root@master test]# kubectl apply -f hpa-nginx.yaml
horizontalpodautoscaler.autoscaling/nginx-hpa created
[root@master test]# kubectl get hpa
NAME        REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx-hpa   Deployment/nginx   7%/50%    1         10        1          59s

更换终端进行测试

#在pod中执行命令，增加内存负载
[root@master ~]# kubectl exec -it nginx-78f4944bb8-2rz7j -- /bin/sh -c 'dd if=/dev/zero of=/tmp/file1'

等待负载上去，然后查看pod数量与内存使用率

[root@master test]# kubectl get hpa
NAME        REFERENCE          TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
nginx-hpa   Deployment/nginx   137%/50%   1         10        1          12m
[root@master test]# kubectl get hpa
NAME        REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx-hpa   Deployment/nginx   14%/50%   1         10        3          12m
[root@master test]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-78f4944bb8-2rz7j   1/1     Running   0          21m
nginx-78f4944bb8-bxh78   1/1     Running   0          34s
nginx-78f4944bb8-g8w2h   1/1     Running   0          34s
#与CPU相同，内存上去了也会自动创建pod

K8S（5）HPA

一、HPA概述

二、HPA版本

三、HPA部署

（1）部署metrics-server

（2）创建Deployment

（3）基于CPU创建HPA

（4）基于内存创建的HPA

热门文章

最新文章

相关课程

相关电子书

相关实验场景

推荐镜像

热门

活动广场

任务中心

开发者评测

高校计划

乘风者计划

训练营

阿里云MVP

话题

直播

下载

镜像站

技术资料

插件

K8S（5）HPA

一、HPA概述

二、HPA版本

三、HPA部署

（1）部署metrics-server

（2）创建Deployment

（3）基于CPU创建HPA

（4）基于内存创建的HPA

热门文章

最新文章

相关课程

相关电子书

相关实验场景

推荐镜像