容器服务kubernetes federation v2实践三：基于ReplicaSchedulingPreference的集群调度

概要

服务部署在多个k8s集群中，往往希望服务跨多个集群进行调度设置。在federation v2中，可以通过ReplicaSchedulingPreference来满足联邦集群的调度设置。下面通过一个简单例子来介绍federation v2中ReplicaSchedulingPreference的用法。

原理分析

federation-controller-manager监听到ReplicaSchedulingPreference变化的时候，通过分析调度配置，生成FederatedDeployment（并且会监听该变化，进行终态保持，即要修改各个集群Deployment只能修改ReplicaSchedulingPreference），然后FederatedDeployment监听到变化会生成各个集群的Deployment。

环境准备

参考阿里云Kubernetes容器服务上体验Federation v2，完成两个集群的join，并配置好kubeconfig, 如下：

kubectl config get-contexts
CURRENT   NAME       CLUSTER    AUTHINFO            NAMESPACE
*         cluster1   cluster1   kubernetes-admin1
          cluster2   cluster2   kubernetes-admin2

部署FederatedDeployment

执行如下yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: test-namespace

---

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedNamespace
metadata:
  name: test-namespace
  namespace: test-namespace
spec:
  placement:
    clusterNames:
    - cluster1
    - cluster2

---

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedDeployment
metadata:
  name: test-deployment
  namespace: test-namespace
spec:
  template:
    metadata:
      labels:
        app: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - image: nginx
            name: nginx
            resources:
              limits:
                cpu: 500m
              requests:
                cpu: 200m
  placement:
    clusterNames:
    - cluster1
    - cluster2

各个集群Deployment情况如下：

kubectl get deployment test-deployment -n test-namespace --context cluster1
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   2         2         2            2           8h

kubectl get deployment test-deployment -n test-namespace --context cluster2
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   2         2         2            2           8h

配置ReplicaSchedulingPreference

apiVersion: scheduling.federation.k8s.io/v1alpha1
kind: ReplicaSchedulingPreference
metadata:
  name: test-deployment
  namespace: test-namespace
spec:
  targetKind: FederatedDeployment
  totalReplicas: 5
  clusters:
    "cluster1":
       weight: 1
       minReplicas: 1
    "cluster2":
       weight: 1
       minReplicas: 1

查看FederatedDeployment详情：

Name:         test-deployment
Namespace:    test-namespace
Labels:       <none>
Annotations:  <none>
API Version:  types.federation.k8s.io/v1alpha1
Kind:         FederatedDeployment
Metadata:
  Creation Timestamp:  2019-05-16T02:39:09Z
  Finalizers:
    federation.k8s.io/sync-controller
  Generation:        1
  Resource Version:  218975248
  Self Link:         /apis/types.federation.k8s.io/v1alpha1/namespaces/test-namespace/federateddeployments/test-deployment
  UID:               c82681b0-7783-11e9-bcd8-e6193977a31d
Spec:
  Overrides:
    Cluster Name:  cluster2
    Cluster Overrides:
      Path:        spec.replicas
      Value:       3
    Cluster Name:  cluster1
    Cluster Overrides:
      Path:   spec.replicas
      Value:  4
  Placement:
    Cluster Names:
      cluster2
      cluster1
  Template:
    Metadata:
      Labels:
        App:  nginx
    Spec:
      Replicas:  2
      Selector:
        Match Labels:
          App:  nginx
      Template:
        Metadata:
          Labels:
            App:  nginx
        Spec:
          Containers:
            Image:  nginx
            Name:   nginx

可以看到FederatedDeployment信息已经被修改，查看各个集群Deployment详情：

kubectl get deployment test-deployment -n test-namespace --context cluster1
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   4         4         4            4           9h

kubectl get deployment test-deployment -n test-namespace --context cluster2
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
test-deployment   3         3         3            3           9h

可以看到通过配置ReplicaSchedulingPreference来配置集群调度策略，当然还有其他的一些属性可以配置，比如每个集群的调度权重、最小副本数和最大副本数。

总结

在Federation v2联邦集群中，可以通过ReplicaSchedulingPreference来配置各个集群的调度策略，统一管理各个集群服务的分布。