实验:Helm部署MySQL服务
//在部署MySQL实例前,可以先查看对应Chart包的信息,需要提前做依赖的环境,当然也可以直接部署,然后一步一步推理、拍错。
//这里我们直接部署安装一次MySQL,安装之前,先将之前的实例删除,使用uninstall命令。
[root@master mysql]# helm uninstall test-mysql
//直接部署安装MySQL实例(通过名为stable的微软源)。
[root@master mysql]# helm install new-mysql stable/mysql
//可以查看安装部署的详细信息。
[root@master mysql]# helm list [root@master mysql]# helm status new-mysql
//首先,确认,都生产了什么资源对象。
[root@master mysql]# ls templates/ configmap.yaml _helpers.tpl pvc.yaml svc.yaml deployment.yaml NOTES.txt secrets.yaml
//通过查看Pod的详细信息,我们判断,由于PVC没有PV支持,所以PVC现在一直处于Pending的状态,导致Pod绑定这个PVC之后,也会出现问题,所以,解决思路是给这PVC创建一个PV。
//这里做基于NFS服务的PV服务,先开启NFS服务。
[root@master ~]# yum install nfs-utils -y [root@master ~]# mkdir /nfsdata [root@master ~]# cat /etc/exports /nfsdata *(rw,sync,no_root_squash) [root@master ~]# systemctl start rpcbind [root@master ~]# systemctl enable rpcbind [root@master ~]# systemctl start nfs-server.service [root@master ~]# systemctl enable nfs-server.service Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. [root@master ~]# showmount -e Export list for master: /nfsdata *
//为PVC创建一个PV.
[root@master templates]# mkdir /nfsdata/new-pv [root@master ~]# vim new-pv.yaml ```yaml apiVersion: v1 kind: PersistentVolume metadata: name: new-pv spec: capacity: storage: 8Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain nfs: path: /nfsdata/new-pv server: 192.168.8.10 kubectl apply -f new-pv.yaml
//查看对应的PV,PVC状态,已经成功绑定。
[root@master ~]# kubectl get pv,pvc NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/new-pv 8Gi RWO Retain Bound default/new-mysql-mysql 25s NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/new-mysql-mysql Bound new-pv 8Gi RWO 24m
//此时,查看Pod,已经正常运行了。
[root@master mysql]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES new-mysql-mysql-6cf95546fb-x7v9h 1/1 Running 0 27m 10.244.1.31 node01
---------------------------------------------
Releases的升级与回滚
//目前releases的镜像版本为:5.7.30版本,假设将此镜像版本升级为5.7.40
[root@master mysql]# helm upgrade --set imageTag=5.7.40 new-mysql stable/mysql [root@master ~]# kubectl get deployments. -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR new-mysql-mysql 1/1 1 1 57m new-mysql-mysql mysql:5.7.40 app=new-mysql-mysql
//helm也有查看历史版本的命令。
[root@master ~]# helm history new-mysql REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1 Fri Sep 23 10:50:53 2022 superseded mysql-1.6.9 5.7.30 Install complete 2 Fri Sep 23 11:06:57 2022 deployed mysql-1.6.9 5.7.40 Upgrade complete
//helm回滚(helm可以回滚到指定版本)
[root@master ~]# helm rollback new-mysql 1 Rollback was a success! Happy Helming! [root@master ~]# kubectl get deployments. -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR new-mysql-mysql 1/1 1 1 62m new-mysql-mysql mysql:5.7.30 app=new-mysql-mysql Usage: helm history RELEASE_NAME [flags] [root@master ~]# helm history new-mysql REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1 Thu Apr 30 14:23:06 2020 superseded mysql-0.3.5 Install complete 2 Thu Apr 30 15:19:01 2020 superseded mysql-0.3.5 Upgrade complete 3 Thu Apr 30 15:24:43 2020 deployed mysql-0.3.5 Rollback to 1
-------------------------------------------------------
//重新部署一个MySQL实例,以自己的名称命名,并且,在部署的时候,要求使用StorageClass资源,并且对应的MYSQL_ROOT_PASSWORD=123.com.
//首先,我们先去创建出来一个SC资源。
1.部署NFS.
2.RBAC的权限。
[root@master ~]# vim rbac-rolebind.yaml ```yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-provisioner
---
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: nfs-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["watch", "create", "update", "patch"] - apiGroups: [""] resources: ["services", "endpoints"] verbs: ["get","create","list", "watch","update"] - apiGroups: ["extensions"] resources: ["podsecuritypolicies"] resourceNames: ["nfs-provisioner"] verbs: ["use"]
---
kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-provisioner subjects: - kind: ServiceAccount name: nfs-provisioner namespace: default roleRef: kind: ClusterRole name: nfs-provisioner-runner apiGroup: rbac.authorization.k8s.io
3.部署nfs-client-provisoner
[root@master ~]# vim nfs-deployment.yaml ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: nfs-client-provisioner spec: selector: matchLabels: app: nfs-client-provisioner replicas: 1 strategy: type: Recreate template: metadata: labels: app: nfs-client-provisioner spec: serviceAccount: nfs-provisioner containers: - name: nfs-client-provisioner image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: bdqn-provisioner - name: NFS_SERVER value: 192.168.8.10 - name: NFS_PATH value: /nfsdata volumes: - name: nfs-client-root nfs: server: 192.168.8.10 path: /nfsdata
4.创建storageclass资源。
[root@master ~]# vim storageclass.yaml ```yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: mysql-sc provisioner: bdqn-provisioner reclaimPolicy: Retain
//根据要求,我们需要更改默认的values.yaml文件,打开storageClass字段,并将上述sc资源的名称写入。
[root@master mysql]# pwd /root/.cache/helm/repository/mysql [root@master mysql]# vim values.yaml ```yaml ## 113 storageClass: "mysql-sc" 114 accessMode: ReadWriteOnce 115 size: 8Gi
//接下来就可以直接部署MySQL实例了,不过还有要求mysql_root_password,这个在README.md文档内有提示,可以直接使用。
[root@master mysql]# helm install bdqn-mysql --set mysqlRootPassword=123.com -f ~/.cache/helm/repository/mysql/values.yaml stable/mysql
//验证服务全部运行之后,可以登录MySQL服务
[root@master mysql]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES bdqn-mysql-mysql-7b89c7b99-mlntq 1/1 Running 0 57s 10.244.1.33 node01 [root@master mysql]# kubectl exec -it bdqn-mysql-mysql-7b89c7b99-mlntq -- mysql -u root -p123.com ... Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
-------------------------------------------------
开发自己的Chart
//创建自定义Chart包。
[root@master helm]# helm create mychart Creating mychart [root@master helm]# ls mychart [root@master helm]# tree -C mychart/ mychart/ ├── charts ├── Chart.yaml ├── templates │ ├── deployment.yaml │ ├── _helpers.tpl │ ├── hpa.yaml │ ├── ingress.yaml │ ├── NOTES.txt │ ├── serviceaccount.yaml │ ├── service.yaml │ └── tests │ └── test-connection.yaml └── values.yaml
3 directories, 10 files
//要求,自己创建的chart包,使用自定义镜像:v1.0版本,replicas:4。然后这个项目单只需要一个Deployment资源和Service(type: NodePort,nodePort: 30010)资源即可。
//根据要求,修改values.yaml文件对应的值。
[root@master mychart]# pwd /opt/helm/mychart [root@master mychart]# vim values.yaml
//关于Deployment资源部分
```yaml
replicaCount: 4 image: repository: nginx pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart version. tag: ""
//关于Service资源部分
```yaml
service: type: NodePort port: 80 nodePort: 30010
//关于禁用hpa、serviceaccount、Ingress资源,直接在values.yaml文件中,选择不启用即可。
```yaml
autoscaling: enabled: false serviceAccount: # Specifies whether a service account should be created create: false ingress: enabled: false
//在安装Chart包之前,可以使用lint 选项对chart包做语法检查。
[root@master helm]# helm lint mychart/ ==> Linting mychart/ [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed
Chart包的几种部署方式
1.安装仓库中的Chart包,
[root@master helm]# helm install new-mysql aliyun/mysql
2.通过helm pull 下来的tar包也是可以安装的。
[root@master repository]# helm install test-redis ./redis-1.1.15.tgz
3.可以通过tar包解压过后的一个完整的目录去安装。
[root@master repository]# helm install new-redis ./redis
4.可以通过URL指定的某个路径下去安装。
~]# helm install test-url https://example.com/charts/nginx-1.2.tgz
PS: 小技巧,自定义的chart目录,也可以使用helm package命令达成tar包。
[root@master helm]# helm package mychart/ Successfully packaged chart and saved it to: /opt/helm/mychart-0.1.0.tgz
//部署自定义Chart包。
[root@master helm]# helm install bdqn mychart -f mychart/values.yaml NAME: bdqn LAST DEPLOYED: Wed May 6 14:43:43 2020 NAMESPACE: default STATUS: deployed REVISION: 1 NOTES: 1. Get the application URL by running these commands: export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services bdqn-mychart) export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}") echo http://NODEIP:NODE_IP:NODE_PORT
//部署完成之后,查看对应的资源生产情况,根据实际的情况,再去做服务的优化,及依赖服务的部署(比如:镜像的私有仓库),
[root@master helm]# kubectl get deployments. NAME READY UP-TO-DATE AVAILABLE AGE bdqn-mychart 4/4 4 4 11m bdqn-mysql-mysql 1/1 1 1 5d23h new-mysql-mysql 1/1 1 1 6d nfs-client-provisioner 1/1 1 1 5d23h [root@master helm]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE bdqn-mychart NodePort 10.96.183.87 80:30774/TCP 11m bdqn-mysql-mysql ClusterIP 10.108.33.139 3306/TCP 5d23h kubernetes ClusterIP 10.96.0.1 443/TCP 24d new-mysql-mysql ClusterIP 10.109.181.209 3306/TCP 6d
PS: 查看对应的资源的情况中,我们发现svc资源,并没有像values.yaml文件中指定的nodePort端口,这是因为,template目录下的service.yaml文件中,并没有对应的nodePort字段,所以默认值引用失败,需要在service.yaml文件中添加。
[root@master templates]# pwd /opt/helm/mychart/templates [root@master templates]# vim service.yaml
```yaml
14
nodePort: {{ .Values.service.nodePort }}
//删除releases再重新部署即可
[root@master templates]# helm uninstall bdqn release "bdqn" uninstalled [root@master helm]# helm install bdqn mychart -f mychart/values.yaml NAME: bdqn ... [root@master helm]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE bdqn-mychart NodePort 10.101.138.143 80:30010/TCP 11m
//注意:如果是自定义的Chart包,进行升级操作,或者说是在部署releases的时候,指定了values.yaml文件的位置了,那么在更新的时候,单纯的使用upgrade命令就做不到完全的更新了,
[root@master helm]# helm upgrade --set imageTag=v2 bdqn mychart Release "bdqn" has been upgraded. Happy Helming! NAME: bdqn LAST DEPLOYED: Wed May 6 15:24:48 2020 NAMESPACE: default STATUS: deployed REVISION: 2 [root@master helm]# kubectl get deployments. -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR bdqn-mychart 4/4 4 4 20m mychart nginx:v1
//这里在对releases进行升级操作是,可以直接指定新的values.yaml文件。
[root@master helm]# vim mychart/values.yaml ... 11 tag: "latest" ... [root@master helm]# helm upgrade bdqn mychart -f mychart/values.yaml Release "bdqn" has been upgraded. Happy Helming! NAME: bdqn LAST DEPLOYED: Wed May 6 15:31:24 2020 NAMESPACE: default STATUS: deployed REVISION: 3 [root@master helm]# kubectl get deployments. -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR bdqn-mychart 3/4 4 3 28m mychart nginx:latest
查看更新历史:
helm history bdqn
回滚第一个版本:
helm rollback bdqn 1
-----------------------------------------------------------
创建chart仓库
任何的HTTP Server都可以作为chart仓库。在node01节点上,部署一个httpd server。
1、在node01节点启动一个httpd服务。
[root@node01 ~]# docker run -d -p 8081:80 -v /var/www:/usr/local/apache2/htdocs httpd
2、通过helm package命令将mycahrt目录达成tar包。
[root@master helm]# helm package mychart/ Successfully packaged chart and saved it to: /opt/helm/mychart-0.1.0.tgz
3、执行helm repo index 生产仓库的index文件。
[root@master helm]# mkdir myrepo [root@master helm]# mv mychart-0.1.0.tgz myrepo/ [root@master helm]# helm repo index myrepo --url http://192.168.8.20:8081/charts [root@master helm]# ls myrepo/ index.yaml mychart-0.1.0.tgz [root@master helm]# vim myrepo/index.yaml apiVersion: v1 entries: mychart: - apiVersion: v2 appVersion: 1.16.0 created: "2020-05-06T16:35:44.42504948+08:00" description: A Helm chart for Kubernetes digest: 67bef10cb7ed0773d62d14dc88562cfd00fa98217b58c7ce190e4bed01a797d0 name: mychart type: application urls: - http://192.168.8.20:8081/charts/mychart-0.1.0.tgz version: 0.1.0 generated: "2020-05-06T16:35:44.423217935+08:00"
PS: Helm会扫描myrepo目录中所有的tgz的包,并生产index.yaml文件。--url 指定的是新的仓库的访问路径。新生产的index.yaml记录了当前仓库中素有的chart信息。
4、将 mychart-0.1.0.tgz包和index.yaml上传到node01:/var/www/charts目录中。
//在node01上创建
[root@node01 ~]# mkdir /var/www/charts
//会到master节点,复制文件。
[root@master myrepo]# pwd /opt/helm/myrepo [root@master myrepo]# scp index.yaml mychart-0.1.0.tgz node01:/var/www/charts index.yaml 100% 419 155.2KB/s 00:00 mychart-0.1.0.tgz 100% 3612 510.6KB/s 00:00
5、通过helm repo add 命令将心仓库添加到Helm.
[root@master helm]# helm repo add newrepo http://192.168.8.20:8081/charts "newrepo" has been added to your repositories [root@master helm]# helm repo list NAME URL aliyun https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts newrepo http://192.168.8.20:8081/charts
6、直接从newrepo中部署服务了。
[root@master helm]# helm search repo mychart NAME CHART VERSION APP VERSION DESCRIPTION newrepo/mychart 0.1.0 1.16.0 A Helm chart for Kubernetes [root@master helm]# helm repo update [root@master helm]# helm uninstall bdqn [root@master helm]# helm install new-http newrepo/mychart
...
7、如果现在,再生产一个自定义Chart包,把它加入到newrepo中。
[root@master helm]# helm create new-chart [root@master helm]# helm package new-chart/ Successfully packaged chart and saved it to: /opt/helm/new-chart-0.1.0.tgz [root@master helm]# mv new-chart-0.1.0.tgz myrepo/ [root@master helm]# helm repo index myrepo --url http://192.168.8.20:8081/charts [root@master helm]# cat myrepo/index.yaml
```yaml
apiVersion: v1 entries: mychart: - apiVersion: v2 appVersion: 1.16.0 created: "2020-05-06T16:58:05.640581343+08:00" description: A Helm chart for Kubernetes digest: 67bef10cb7ed0773d62d14dc88562cfd00fa98217b58c7ce190e4bed01a797d0 name: mychart type: application urls: - http://192.168.8.20:8081/charts/mychart-0.1.0.tgz version: 0.1.0 new-chart: - apiVersion: v2 appVersion: 1.16.0 created: "2020-05-06T16:58:05.643010881+08:00" description: A Helm chart for Kubernetes digest: a1ee510a357a9d2308c1f837833232ce4f822903d82ad1de834d67de43e07312 name: new-chart type: application urls: - http://192.168.8.20:8081/charts/new-chart-0.1.0.tgz version: 0.1.0 generated: "2020-05-06T16:58:05.639022589+08:00"
[root@master helm]# scp myrepo/* node01:/var/www/charts index.yaml 100% 781 132.9KB/s 00:00 mychart-0.1.0.tgz 100% 3612 863.0KB/s 00:00 new-chart-0.1.0.tgz 100% 3580 1.3MB/s 00:00 [root@master helm]# helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "newrepo" chart repository ...Successfully got an update from the "aliyun" chart repository Update Complete. ⎈ Happy Helming!⎈ [root@master helm]# helm search repo new-chart NAME CHART VERSION APP VERSION DESCRIPTION newrepo/new-chart 0.1.0 1.16.0 A Helm chart for Kubernetes