容器服务kubernetes federation v2实践一：基于External-DNS的多集群Ingress DNS实践

概要

External-DNS提供了编程方式管理Kubernetes Ingress资源的DNS的功能，方便用户从Ingress管理DNS解析记录。而在kubernetes federation v2环境中，使用External-DNS可以快速的管理多个联邦集群的Ingress DNS解析，降低用户的操作成本。下面将简单介绍在阿里云容器服务环境中，如何使用External-DNS管理联邦集群的Ingress DNS解析。

联邦集群准备

参考阿里云Kubernetes容器服务上体验Federation v2 搭建两个集群组成的联邦集群（配置好kubeconfig，并完成两个集群的join）。

配置RAM信息

选择Kubernetes集群节点列表内任意一个Worker节点，打开对应的节点列表信息页面。

找到对应的 RAM 角色，打开RAM控制台，找到对应的角色名称，添加【AliyunDNSFullAccess】权限。

注意：每个集群都需要配置RAM信息。

部署External-DNS

配置RBAC

执行下面yaml：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get","watch","list"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get","watch","list"]
- apiGroups: ["extensions"]
  resources: ["ingresses"]
  verbs: ["get","watch","list"]
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["list"]
- apiGroups: ["multiclusterdns.federation.k8s.io"]
  resources: ["dnsendpoints"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
- kind: ServiceAccount
  name: external-dns
  namespace: default

部署External-DNS服务

执行下面yaml：

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: external-dns
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
      - name: external-dns
        image: registry.cn-beijing.aliyuncs.com/acs/external-dns:v0.5.8-27
        args:
        - --source=crd
        - --crd-source-apiversion=multiclusterdns.federation.k8s.io/v1alpha1
        - --crd-source-kind=DNSEndpoint
        - --provider=alibabacloud
        - --policy=sync # enable full synchronization
        - --registry=txt
        - --txt-prefix=cname
        - --txt-owner-id=my-identifier
        - --alibaba-cloud-config-file= # enable sts token
        volumeMounts:
        - mountPath: /usr/share/zoneinfo
          name: hostpath
      volumes:
      - name: hostpath
        hostPath:
          path: /usr/share/zoneinfo
          type: Directory

部署验证资源

创建FederatedDeployment和FederatedService：

apiVersion: v1
kind: Namespace
metadata:
  name: test-namespace

---

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedNamespace
metadata:
  name: test-namespace
  namespace: test-namespace
spec:
  placement:
    clusterNames:
    - cluster1
    - cluster2

---

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedDeployment
metadata:
  name: test-deployment
  namespace: test-namespace
spec:
  template:
    metadata:
      labels:
        app: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - image: nginx
            name: nginx
            resources:
              limits:
                cpu: 500m
              requests:
                cpu: 200m
  placement:
    clusterNames:
    - cluster1
    - cluster2
    
---

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedService
metadata:
  name: test-service
  namespace: test-namespace
spec:
  template:
    spec:
      selector:
        app: nginx
      type: ClusterIP
      ports:
        - name: http
          port: 80
  placement:
    clusterNames:
    - cluster2
    - cluster1

各个集群ingress创建信息如下：

kubectl get ingress -n test-namespace --context cluster1
NAME           HOSTS   ADDRESS        PORTS   AGE
test-ingress   *       47.93.69.121   80      54m

kubectl get ingress -n test-namespace --context cluster2
NAME           HOSTS   ADDRESS         PORTS   AGE
test-ingress   *       39.106.232.23   80      54m

创建FederatedIngress和IngressDNSRecord

apiVersion: types.federation.k8s.io/v1alpha1
kind: FederatedIngress
metadata:
  name: test-ingress
  namespace: test-namespace
spec:
  template:
    spec:
       backend:
          serviceName: test-service
          servicePort: 80
  placement:
    clusterNames:
    - cluster2
    - cluster1        

---

apiVersion: multiclusterdns.federation.k8s.io/v1alpha1
kind: IngressDNSRecord
metadata:
  name: test-ingress
  namespace: test-namespace
spec:
  hosts:
  - ingress-example.example-domain.club
  recordTTL: 600
    

其中【ingress-example.example-domain.club】为测试阿里云托管的域名，请提前在阿里云上购买域名，并注意替换。

DNS解析验证

dig +short @dns7.hichina.com ingress-example.example-domain.club
47.93.69.121
39.106.232.23

可以看到我们绑定的域名已经解析到了cluster1和cluster2的ingress IP上了。
访问域名相应的服务：

curl ingress-example.sigma-host.club
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

总结

通过上面介绍，可以看到使用External-DNS可以非常方便的管理federation-v2环境下的Ingress DNS解析。