一 背景
利用Gitlab CI实现基础设施编排自动化，用户后续针对基础设施的管理使用Gitlab完成，提交基础设施变更后，会出发pr进行Gitlab CI流水线执行，从而实现基础设施DevOPS流程。
二 流程架构
2.1 架构图
2.2 流程
运维研发编写目标云的基于Terraform的资源清单文件，同事项目内管理Gitlab CI流程，在K8s不同NS下注册有对应的runner，当在不同分支下可以触发不同ns下的CI流程。
开发或运维人员提交代码。
部署在对应名称空间下的runner执行流程，创建运行单个Stage的POD来运行Terraform对应命令，例如init/fmt/play/apply等。
如果要对云上资源进行变更，修改代码，再次提交pr，出发更新流水线。
如果需要销毁，根据CI文件配置提交BUILD为destroy，触发云上销毁动作。
三 预置条件
Gitlab 服务器
注册有项目的gitlab-runner
K8s集群
腾讯云AK账号
四 配置
4.1 Gitlab CI配置
4.1.1 .gitlab.yaml
variables:
# PHASE: BUILD|DESTROY
PHASE: DESTROY
PROXY: http://squiduser:xxzx789@43.134.199.162:3128
PROXY: http://squiduser:xxzx789@43.154.230.17:3128
REGION: "ap-guangzhou"
PLAN_JSON: plan.json
BACKEND_CONF: "backend_oss.conf"
before_script:
- apk add --no-cache curl git jq
- apk add --no-cache jq
- export http_proxy=${SQUID_PROXY}
- export https_proxy=${SQUID_PROXY}
- export TENCENTCLOUD_SECRET_KEY=${TENCENTCLOUD_SECRET_KEY}
- export TENCENTCLOUD_SECRET_ID=${TENCENTCLOUD_SECRET_ID}
- export TF_REGISTRY_CLIENT_TIMEOUT=120000
- export CHECKPOINT_TIMEOUT=500000
- export TF_REGISTRY_DISCOVERY_RETRY=5
- alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'"
配置缓存
cache:
paths:
- ${CI_PROJECT_DIR}/.terraform/*
AI 代码解读
stages:
- init
- validate
- plan
- deploy
Init:
image:
name: hashicorp/terraform:0.14.0
entrypoint: [""]
AI 代码解读
stage: init
retry:
max: 2
when:
- script_failure
AI 代码解读
tags:
- gitlab-runner-k8s-new
AI 代码解读
script:
- terraform version
- terraform init -backend-config=${BACKEND_CONF}
AI 代码解读
only:
- dev
AI 代码解读
Validate:
image:
name: hashicorp/terraform:0.14.0
entrypoint: [""]
AI 代码解读
stage: validate
tags:
- gitlab-runner-k8s-new
AI 代码解读
retry: 2
script:
- terraform init -backend-config=${BACKEND_CONF}
- terraform validate
- terraform fmt -check -recursive || echo 0
AI 代码解读
cache:
paths:
- ${CI_PROJECT_DIR}/.terraform/*
policy: pull
AI 代码解读
allow_failure: true
Plan:
image:
name: hashicorp/terraform:0.14.0
entrypoint: [""]
AI 代码解读
stage: plan
retry: 2
tags:
- gitlab-runner-k8s-new
AI 代码解读
artifacts:
paths:
- plan.bin
- app_config.zip
expire_in: 2 week
AI 代码解读
script:
- terraform init -backend-config=${BACKEND_CONF}
- terraform plan -input=false -out=plan.bin -var region=${REGION}
- terraform show --json "plan.bin" | convert_report > ${PLAN_JSON}
- cat ${PLAN_JSON}
AI 代码解读
only:
variables:
- $PHASE == "BUILD"
AI 代码解读
Apply:
image:
name: hashicorp/terraform:0.14.0
entrypoint: [""]
AI 代码解读
when: manual
stage: deploy
retry: 2
tags:
- gitlab-runner-k8s-new
AI 代码解读
script:
- terraform init -backend-config=${BACKEND_CONF}
- terraform apply -auto-approve -input=false plan.bin
AI 代码解读
only:
variables:
- $PHASE == "BUILD"
AI 代码解读
environment:
name: snunv
AI 代码解读
Destroy:
image:
name: hashicorp/terraform:0.14.0
entrypoint: [""]
AI 代码解读
stage: deploy
retry: 2
tags:
- gitlab-runner-k8s-new
AI 代码解读
script:
- terraform init -backend-config=${BACKEND_CONF}
- terraform destroy -auto-approve -var region=${REGION}
AI 代码解读
only:
variables:
- $PHASE == "DESTROY"
AI 代码解读
4.1.2 环境配置
利用Gitlab CI/CD的Environment进行环境管理。
4.2 Terraform资源
provider "tencentcloud" {
region = var.region
}
terraform {
required_providers {
tencentcloud = {
source = "registry.terraform.io/tencentcloudstack/tencentcloud"
version = ">=1.61.5"
}
AI 代码解读
}
backend "cos" {}
}
输入变量
variable "region" {
type = string
}
再次仅为一个查询示例
data "tencentcloud_instances" "cvm" {
}
输出
output "result" {
value = {
cvm_result = { for k, v in data.tencentcloud_instances.cvm : k => v },
count = data.tencentcloud_instances.cvm.instance_list[*]
AI 代码解读
}
}
为了terraform后端backend安全，将其存储为单独文件，可不同分支或环境进行修改
region = "ap-beijing"
bucket = "tfproject-1253329830"
prefix = "samxxxxitlab/dexxxxxt"
