容器开启数据服务之旅系列（四）

Kubernetes QoS 助力在线运用与大数据离线运用的带宽控制和磁盘控制

概述

本文是2018年大数据峰会上的一些分享，关于在线业务，离线业务在ACK（阿里云容器服务Kubernetes）的平台上通过对bandwidth, disk quota的灵活组合完成在线，离线业务场景的混合部署，来提高总体资源的使用率，以及带宽，本地盘资源的动态分配调整，来控制离线，在线资源水位。

离线大数据运算带宽控制

QoS for bandwidth

annotations:
    kubernetes.io/egress-bandwidth: 10M
    kubernetes.io/ingress-bandwidth: 5M

命名空间隔离和流量资源控制

ingress-bandwidth: 5M
egress-bandwidth: 10M

Pod级别的流量控制

ingress-bandwidth: 10M
egress-bandwidth: 10M

Name:         batch
Labels:       <none>
Annotations:  kubernetes.io/egress-bandwidth=10M
              kubernetes.io/ingress-bandwidth=5M
Status:       Active

Resource Quotas
 Name:                   quota
 Resource                Used   Hard
 --------                ---    ---
 configmaps              0      100
 cpu                     100m   4
 memory                  256Mi  32Gi
 persistentvolumeclaims  0      100
 pods                    1      100
 replicationcontrollers  0      10
 requests.storage        0      1024G
 secrets                 1      100
 services                0      10

Resource Limits
 Type       Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
 ----       --------  ---  ---  ---------------  -------------  -----------------------
 Container  cpu       -    -    100m             1              -
 Container  memory    -    -    256Mi            16Gi           -![image]

带宽限制测试

[ 4] 0.00-10.00 sec 93.7 MBytes 78.6 Mbits/sec receiver

离线大数据运算本地磁盘用量控制

QoS for rootfs

limits:
  ephemeral-storage: 100Mi
requests:
  ephemeral-storage: 100Mi

命名空间隔离和rootfs disk资源控制

limits:
  - default:
      cpu: 1
      memory: 16Gi
      ephemeral-storage: 100Mi
    defaultRequest:
      cpu: 100m
      memory: 256Mi
      ephemeral-storage: 100Mi
    type: Container

Pod级别的rootfs disk控制

limits:
  ephemeral-storage: 100Mi
requests:
  ephemeral-storage: 100Mi

本地磁盘用量限制

apiVersion: v1
kind: LimitRange # set default request/limit for your containers
metadata:
  name: limits
#  namespace: users-namespace # specify your namespace
spec:
  limits:
  - default:
      cpu: 1
      memory: 16Gi
      ephemeral-storage: 100Mi
    defaultRequest:
      cpu: 100m
      memory: 256Mi
      ephemeral-storage: 100Mi
    type: Container     -

在容器内部测试本地盘限制