使用 Terraform 创建托管版 Kubernetes

目前，阿里云容器服务已经可以创建托管版 Kubernetes 集群了。相比于默认的 Kubernetes 集群，托管版本会主动替您运维一套高可用的 Master 组件，免去了默认版本集群中三个 Master ECS 节点，从而节约所需的资金成本及维护时的人力成本。在容器服务控制台，我们为您提供了便捷使用的可视界面一步一步引导式地创建该类型集群。但当您需要反复创建托管版集群、大批量创建集群，或者您就是天生抗拒控制台手工操作的那一类人，可以了解并尝试使用一下 Terraform 了。

Terraform 是一款 Infrastructure as Code 的工具，可以将云端资源代码化。关于 Terraform 的基本介绍本文不再赘述，有兴趣的同学可以参考 《云生态下的基础架构资源管理利器Terraform》 等云栖社区的优秀文章。目前我们一直在支持阿里云 Terraform Provider，已经实现了阿里云上面绝大部分的云产品的对接。

在 2018 年圣诞节来临之前，阿里云 Terraform Provider 已经发布 v1.26.0 版本，其中已经支持了创建托管版Kubernetes 集群，下面我们来一起看下如何实现命令行快速部署一个这样的集群。

创建托管版 Kubernetes 集群

首先我们打开《阿里云 Terraform Provider 文档 - 托管版 Kubernetes》的帮助文档，可以看到该资源 Resource 提供的参数列表。参数分为入参 Argument 和出参 Attributes。入参列表内包含了必填参数以及可选参数，例如 name 和 name_prefix 就是一对必填参数，但它们互斥，即不能同时填写。如果填了 name，集群名就是 name 的值，如果填了 name_prefix，集群名会以 name_prefix 开头自动生成一个。我们对照文档中的参数列表 Argument Reference，先草拟出一个集群的描述，为了方便起见，我把填写每个参数的理由都注释在代码中。

# 引入阿里云 Terraform Provider
provider "alicloud" {
  # 填入您的账号 Access Key
  access_key = "FOO"
  # 填入您的账号 Secret Key
  secret_key = "BAR"
  # 填入想创建的 Region
  region     = "cn-hangzhou"
  # 可选参数，默认不填就使用最新版本
  version    = "v1.26.0"
}

# 必要的资源标识
# alicloud_cs_managed_kubernetes 表明是托管版 Kubernetes 集群
# k8s 代表该资源实例的名称
resource "alicloud_cs_managed_kubernetes" "k8s" {
  # 集群名称，可以带中划线，一个账户内的集群名称不能相同
  name = "test-managed-kubernetes"
  # 可以从 ECS 控制台上面查询到可用区信息，以及对应的 ECS 实例类型库存
  # 以下代表 Worker 节点将部署在 cn-hangzhou-h 这个可用区，采用 ecs.c5.xlarge 这个机型。
  availability_zone = "cn-hangzhou-h"
  worker_instance_types = ["ecs.c5.xlarge"]
  # 配置该集群 Worker 节点数为 2 个，该数字后续可以再扩容
  worker_numbers = [2]
  # Worker 节点使用高效云盘
  worker_disk_category  = "cloud_efficiency"
  # 默认为 true，会在 VPC 内创建一个 Nat 网关用于 ECS 连上互联网
  new_nat_gateway = true
  # 配置所有 ECS 的默认 Root 密码，此处也可以用密钥对 key_name 代替，但需要提前创建
  password = "Test12345"
  # Kubernetes 集群内所有 Pod 使用的子网网段，不能与 service_cidr 和 ECS 所在网段冲突
  # 默认创建的 VPC 是 192.168.0.0/16 这个网段内的，所以 pod_cidr 和 service_cidr 可以使用 172 网段
  # 请参考 VPC下 Kubernetes 的网络地址段规划
  pod_cidr = "172.20.0.0/16"
  service_cidr = "172.21.0.0/20"
  # 安装云监控插件
  install_cloud_monitor = true
}

我们可以将以上的配置保存为一个 main.tf 描述文件，在该文件的当前目录下执行 terraform init 和 terraform apply。

xh4n3@xh4n3:~/ops/terraform-example% terraform init --get-plugins=true -upgrade

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "alicloud" (1.26.0)...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

xh4n3@xh4n3:~/ops/terraform-example% terraform apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + alicloud_cs_managed_kubernetes.k8s
      id:                          <computed>
      availability_zone:           "cn-hangzhou-h"
      install_cloud_monitor:       "true"
      name:                        "test-managed-kubernetes"
      name_prefix:                 "Terraform-Creation"
      new_nat_gateway:             "true"
      password:                    <sensitive>
      pod_cidr:                    "172.20.0.0/16"
      security_group_id:           <computed>
      service_cidr:                "172.21.0.0/20"
      vpc_id:                      <computed>
      vswitch_ids.#:               <computed>
      worker_disk_category:        "cloud_efficiency"
      worker_disk_size:            "40"
      worker_instance_charge_type: "PostPaid"
      worker_instance_types.#:     "1"
      worker_instance_types.0:     "ecs.c5.xlarge"
      worker_nodes.#:              <computed>
      worker_numbers.#:            "1"
      worker_numbers.0:            "2"


Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

从上述日志中可以看到，terraform init 会把我们用到的 Provider 插件下载好，terraform apply 会根据我们的 main.tf 描述文件计算出需要执行的操作，上述显示将会创建一个 alicloud_cs_managed_kubernetes.k8s 的资源，需要我们输入 yes 来确认创建。确认创建后，创建大约会耗时五分钟，terraform 会输出类似下面的日志。

# 以上省略
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_cs_managed_kubernetes.k8s: Creating...
  availability_zone:           "" => "cn-hangzhou-h"
  install_cloud_monitor:       "" => "true"
  name:                        "" => "test-managed-kubernetes"
  name_prefix:                 "" => "Terraform-Creation"
  new_nat_gateway:             "" => "true"
  password:                    "<sensitive>" => "<sensitive>"
  pod_cidr:                    "" => "172.20.0.0/16"
  security_group_id:           "" => "<computed>"
  service_cidr:                "" => "172.21.0.0/20"
  vpc_id:                      "" => "<computed>"
  vswitch_ids.#:               "" => "<computed>"
  worker_disk_category:        "" => "cloud_efficiency"
  worker_disk_size:            "" => "40"
  worker_instance_charge_type: "" => "PostPaid"
  worker_instance_types.#:     "" => "1"
  worker_instance_types.0:     "" => "ecs.c5.xlarge"
  worker_nodes.#:              "" => "<computed>"
  worker_numbers.#:            "" => "1"
  worker_numbers.0:            "" => "2"
alicloud_cs_managed_kubernetes.k8s: Still creating... (10s elapsed)
alicloud_cs_managed_kubernetes.k8s: Still creating... (20s elapsed)
alicloud_cs_managed_kubernetes.k8s: Still creating... (30s elapsed)
# 以上省略
alicloud_cs_managed_kubernetes.k8s: Creation complete after 6m5s (ID: cc54df7d990a24ed18c1e0ebacd36418c)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

当出现 Apply complete! Resources: 1 added 字样的时候，集群已经成功创建，此时我们也可以登录控制台后在控集群列表中看到集群。

修改托管版 Kubernetes 集群

在 Terraform Provider 中，我们提供了一部分参数的修改能力，一般情况下，所有非 Force New Resouce（强制新建资源）的参数都可以被修改。下面我们修改部分参数，注释内容为更新的项目。

provider "alicloud" {
  access_key = "FOO"
  secret_key = "BAR"
  region     = "cn-hangzhou"
  version    = "v1.26.0"
}

resource "alicloud_cs_managed_kubernetes" "k8s" {
  # 更换集群的名称为 test-managed-kubernetes-updated
  name = "test-managed-kubernetes-updated"
  availability_zone = "cn-hangzhou-h"
  worker_instance_types = ["ecs.c5.xlarge"]
  # 修改 worker_numbers 为 3，可以扩容一个 worker 节点
  worker_numbers = [3]
  worker_disk_category  = "cloud_efficiency"
  new_nat_gateway = true
  password = "Test12345"
  pod_cidr = "172.20.0.0/16"
  service_cidr = "172.21.0.0/20"
  install_cloud_monitor = true
  # 导出集群的连接配置文件到 /tmp 目录
  kube_config = "/tmp/config"
  # 导出集群的证书相关文件到 /tmp 目录，下同
  client_cert = "/tmp/client-cert.pem"
  client_key = "/tmp/client-key.pem"
  cluster_ca_cert = "/tmp/cluster-ca-cert.pem"
}

同创建集群一样，修改集群时使用的命令也是 terraform apply。执行后我们得到以下日志输出，输入 yes 并回车，我们就可以把该集群的名称改为 test-managed-kubernetes-updated，worker 节点扩容至 3 节点，同时将导出证书和连接文件到本机的 /tmp 目录。

xh4n3@xh4n3:~/ops/terraform-example% terraform apply
alicloud_cs_managed_kubernetes.k8s: Refreshing state... (ID: cc54df7d990a24ed18c1e0ebacd36418c)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ alicloud_cs_managed_kubernetes.k8s
      client_cert:      "" => "/tmp/client-cert.pem"
      client_key:       "" => "/tmp/client-key.pem"
      cluster_ca_cert:  "" => "/tmp/cluster-ca-cert.pem"
      kube_config:      "" => "/tmp/config"
      name:             "test-managed-kubernetes" => "test-managed-kubernetes-updated"
      worker_numbers.0: "2" => "3"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_cs_managed_kubernetes.k8s: Modifying... (ID: cc54df7d990a24ed18c1e0ebacd36418c)
  client_cert:      "" => "/tmp/client-cert.pem"
  client_key:       "" => "/tmp/client-key.pem"
  cluster_ca_cert:  "" => "/tmp/cluster-ca-cert.pem"
  kube_config:      "" => "/tmp/config"
  name:             "test-managed-kubernetes" => "test-managed-kubernetes-updated"
  worker_numbers.0: "2" => "3"
alicloud_cs_managed_kubernetes.k8s: Still modifying... (ID: cc54df7d990a24ed18c1e0ebacd36418c, 10s elapsed)
alicloud_cs_managed_kubernetes.k8s: Still modifying... (ID: cc54df7d990a24ed18c1e0ebacd36418c, 20s elapsed)
alicloud_cs_managed_kubernetes.k8s: Still modifying... (ID: cc54df7d990a24ed18c1e0ebacd36418c, 30s elapsed)
# 以上省略
alicloud_cs_managed_kubernetes.k8s: Modifications complete after 4m4s (ID: cc54df7d990a24ed18c1e0ebacd36418c)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Terraform apply 运行成功后，控制台中显示的集群信息已经表明现在集群已经变成了我们期望的状态。在本机上，我们也通过导出的连接文件，用 kubectl 连接到集群。

附录

控制台创建托管版 Kubernetes 集群帮助文档
https://help.aliyun.com/document_detail/95108.html
云生态下的基础架构资源管理利器Terraform
https://yq.aliyun.com/articles/215592
阿里云 Terraform Provider 代码库
https://github.com/terraform-providers/terraform-provider-alicloud
阿里云 Terraform Provider 文档
https://www.terraform.io/docs/providers/alicloud/index.html
阿里云 Terraform Provider 文档 - 托管版 Kubernetes
https://www.terraform.io/docs/providers/alicloud/r/cs_managed_kubernetes.html
VPC下 Kubernetes 的网络地址段规划
https://help.aliyun.com/document_detail/86500.html
Terraform 部署容器服务Kubernetes集群及Wordpress应用
https://yq.aliyun.com/articles/641627