基于 Kubernetes 实现 CI/CD 配置
一、基本介绍
二、基于 Kubernetes 实现 CI/CD 配置
1.配置 GitLab
2.配置 Jenkins
3.实现 CI/CD 配置
4.验证
一、基本介绍
基于 Kubernetes 实现 CI/CD 配置，其实和往常那些 CI/CD 配置并没有太大区别。都是通过 提交代码，拉取代码，构建代码，发布代码来实现的。 只不过要是通过 K8s 来实现的话，则是需要将构建好的代码打包成镜像，通过镜像的方式来运行。
CI/CD 流程图：
开发将代码提交代码仓库后，我们便可以通过在 Jenkins 上配置脚本或是 Pipline 的方式来实现代码发布，其中发布有两种方式，一种是通过手动发布，另外一种可以通过 WebHook 插件来实现提交代码便自动发布（生产环境不建议自动发布）
- 脚本内容一般分为：克隆代码、编译代码、将编译好的代码打包成镜像、运行镜像几个步骤。
二、基于 Kubernetes 实现 CI/CD 配置
- 下面我们是通过容器的方式安装配置，物理安装可以看我前面的文章：传送门
1.配置 GitLab
1）安装 Docker-Compose
[root@k8s-master01 ~]# wget "https://github.com/docker/compose/releases/download/v2.3.2/docker-compose-$(uname -s)-$(uname -m)" -O /usr/local/bin/docker-compose [root@k8s-master01 ~]# chmod +x /usr/local/bin/docker-compose [root@k8s-master01 ~]# docker-compose --version
2）安装 GitLab
[root@k8s-master01 ~]# vim docker-compose.yml version: '3' services: web: image: 'gitlab/gitlab-ce:14.8.5-ce.0' restart: always hostname: 192.168.1.1 environment: GITLAB_OMNIBUS_CONFIG: | external_url 'http://192.168.1.1' ports: - '1080:80' - '1443:443' - '1022:22' volumes: - '/app/gitlab/config:/etc/gitlab' - '/app/gitlab/logs:/var/log/gitlab' - '/app/gitlab/data:/var/opt/gitlab' [root@k8s-master01 ~]# docker-compose up -d
- 因为博主的电脑配置不是很高，所以就不使用上面的方式安装 GitLab，而是直接使用 GitHub 上面的仓库。
2.配置 Jenkins
1）安装 NFS 存储，并配置共享目录
[root@k8s-master01 ~]# yum -y install nfs-utils rpcbind [root@k8s-master01 ~]# echo "/app/jenkins *(rw,sync,no_root_squash)" > /etc/exports [root@k8s-master01 ~]# mkdir /app/jenkins [root@k8s-master01 ~]# systemctl start rpcbind nfs
2）创建 PV 和 PVC
[root@k8s-master01 ~]# vim jenkins-pv.yaml apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-pv spec: capacity: storage: 10Gi accessModes: - ReadWriteMany nfs: server: 192.168.1.1 path: /app/jenkins --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jenkins-pvc spec: resources: requests: storage: 10Gi accessModes: - ReadWriteMany [root@k8s-master01 ~]# kubectl create -f jenkins-pv.yaml
2）创建 RBAC 授权
[root@k8s-master01 ~]# vim jenkins-sa.yaml apiVersion: v1 kind: ServiceAccount metadata: name: jenkins-sa --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: jenkins-cr rules: - apiGroups: ["extensions","apps"] resources: ["deployments"] verbs: ["create","delete","get","list","watch","patch","update"] - apiGroups: [""] resources: ["services"] verbs: ["create","delete","get","list","watch","patch","update"] - apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create","delete","get","list","patch","update"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","update"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: jenkins-crb roleRef: kind: ClusterRole name: jenkins-cr apiGroup: rbac.authorization.k8s.io subjects: - kind: ServiceAccount name: jenkins-sa namespace: default [root@k8s-master01 ~]# kubectl create -f jenkins-sa.yaml
3）创建 StatefulSet
[root@k8s-master01 ~]# vim jenkins-statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: jenkins spec: serviceName: jenkins replicas: 1 selector: matchLabels: app: jenkins template: metadata: name: "jenkins" labels: app: jenkins spec: serviceAccountName: jenkins-sa containers: - name: jenkins image: jenkins/jenkins:lts imagePullPolicy: IfNotPresent ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - name: jenkins mountPath: /var/jenkins_home volumes: - name: jenkins persistentVolumeClaim: claimName: jenkins-pvc [root@k8s-master01 ~]# chown -R 1000 /app/jenkins [root@k8s-master01 ~]# kubectl create -f jenkins-statefulset.yaml
4）创建 Service
[root@k8s-master01 ~]# vim jenkins-svc.yaml apiVersion: v1 kind: Service metadata: name: jenkins spec: type: NodePort ports: - name: http port: 8080 targetPort: 8080 nodePort: 30080 - name: agent port: 50000 targetPort: 50000 nodePort: 30090 selector: app: jenkins [root@k8s-master01 ~]# kubectl create -f jenkins-svc.yaml
5）配置 Jenkins
[root@k8s-master01 ~]# cat /app/jenkins/secrets/initialAdminPassword a303d66e915e4ee5b26648a64fdff4be
我们这里安装推荐的插件即可，后面有需求可以再进行安装
3.实现 CI/CD 配置
1）在 Jenkins 宿主机上创建 SSH 密钥
[root@k8s-master01 ~]# ssh-keygen -t rsa # 三连回车 [root@k8s-master01 ~]# cat ~/.ssh/id_rsa.pub # 查看公钥
2）将公钥上传到 GitLab 上
3）将仓库克隆到本地
[root@k8s-master01 ~]# git clone git@github.com:ChenZhuang1217/test.git
4）编写 Go 代码
[root@k8s-master01 ~]# cd test [root@k8s-master01 test]# vim main.go package main import ( "fmt" "net/http" ) func HelloHandler(w http.ResponseWriter, r *http.Request) { fmt.Fprintf(w, "Hello World") } func main() { http.HandleFunc("/", HelloHandler) http.ListenAndServe(":8080", nil) }
5）编写 Dockerfile
[root@k8s-master01 test]# vim Dockerfile FROM golang:1.16 as builder ENV GO111MODULE=on \ GOPROXY=https://goproxy.cn,direct WORKDIR /app COPY . . RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o main main.go FROM busybox:1.28.4 WORKDIR /app COPY --from=builder /app/ . EXPOSE 8080 CMD ["./main"] [root@k8s-master01 test]# docker build -t test-web-server:devops-$(date +%Y-%m-%d-%H-%M-%S) .
6）提交代码（Git 教程可以看博主前面写的文章：Git）
[root@k8s-master01 test]# git add . # 提交到暂存区 [root@k8s-master01 test]# git config --global user.email "Zhuang_zz1217@163.com" # 配置用户邮箱 [root@k8s-master01 test]# git commit -m "This is test CI/CD" # 提交到本地仓库 [root@k8s-master01 test]# git push # 推送到远程仓库
7）创建 Deployment 和 Service
[root@k8s-master01 ~]# vim test-web-server.yaml apiVersion: apps/v1 kind: Deployment metadata: name: test-web-server spec: replicas: 1 selector: matchLabels: app: test-web-server template: metadata: labels: app: test-web-server spec: containers: - name: test-web-server image: test-web-server:devops-2022-04-25-17-16-54 imagePullPolicy: IfNotPresent ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: test-web-server spec: type: NodePort ports: - name: test-web-server port: 8080 targetPort: 8080 nodePort: 30188 selector: app: test-web-server [root@k8s-master01 ~]# kubectl create -f test-web-server.yaml
8）编写 Jenkins 发版脚本
[root@k8s-master01 ~]# vim test.sh #!/bin/bash # 固定时间格式 Second=$(date +%Y-%m-%d-%H-%M-%S) # 备份旧的镜像 Image=$(kubectl -s https://192.168.1.1:6443 describe pod | grep Image: | awk '{print $2}' | grep test) echo $Image > /opt/test-image-$Second # 克隆代码 cd /root if [ -d test ]; then mv test /opt/test-devops-$Second git clone git@github.com:ChenZhuang1217/test.git else git clone git@github.com:ChenZhuang1217/test.git fi # 发布新的镜像 cd /root/test && docker build -t test-web-server:devops-$Second . # 上传到镜像仓库 if [ $? -eq 0 ]; then docker tag test-web-server:devops-$dateImage harbor.tianya.com:5000/test-web-server:devops-$Second docker push harbor.tianya.com:5000/test-web-server:devops-$Second else exit 1 # 退出 (防止运行下面命令) fi # 替换镜像 sed -i 's/image:.*/image: harbor.tianya.com:5000\/test-web-server:devops-'$Second'/g' /root/test-web-server.yaml # 重启应用 kubectl delete -f /root/test-web-server.yaml kubectl create -f /root/test-web-server.yaml [root@k8s-master01 ~]# chmod +x test.sh
上面这个脚本有两步需要注意：
- 上传到镜像仓库： 如果你们没有自己的镜像仓库，可以选择调整脚本或看博主前面写的文章来安装 Harbor 仓库。
- 替换镜像： 我们上面配置的脚本是针对单个模块的，多个模块可以根据 for 循环来实现。
4.验证
1）在 Jenkins 上安装 SSH 插件
- 安装 SSH 插件的原因是因为，我们这个 Jenkins 是容器安装的，而脚本是在宿主机写的，所以通过远程到宿主机来运行脚本。
2）配置远程主机的用户名和密码
3）创建 Jenkins 私钥凭证（类型选择：SSH Username with private key）
4）配置 Jenkins 流水线
5）修改代码
6）在 Jenkins 上发布
