前言：
在实际的生产中，我们可能会有许多的由开发制作的docker镜像，这也就造成使用这些镜像需要打包成tar文件，然后上传到服务器内然后在导入并使用，但，kubernetes节点很多，有时候并不是明确的要在哪个节点使用，因此需要每个节点都上传或者是在部署的时候做节点亲和。那么，很明显这样的方法是不够优雅的。
那么，我们将需要的镜像统一上传到一个自己搭建的私有镜像仓库是一个比较好的解决方案，只需要在部署阶段指定使用私有镜像仓库就可以解决了，并且harbor这样的私有镜像仓库还有安全漏洞扫描功能，能够提升整个系统的安全性。
下面，将就如何在kubernetes集群内使用私有镜像仓库做一个简单的示例。
一，
在kubernetes内使用私有镜像仓库之前，我们需要先有一个私有镜像仓库，并保证这个仓库是可用的。
本文的私有镜像仓库是harbor仓库，该仓库搭建在服务器IP为192.168.217.23。
具体的搭建过程请移步上一篇文章；harbor仓库的构建及简单使用（修订版）_晚风_END的博客-CSDN博客
二，
检查私有镜像仓库是否可用
[root@node3 manifests]# systemctl status harbor ● harbor.service - Harbor Loaded: loaded (/usr/lib/systemd/system/harbor.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2022-12-01 11:43:31 CST; 1h 22min ago Docs: http://github.com/vmware/harbor Main PID: 2690 (docker-compose) Memory: 41.6M CGroup: /system.slice/harbor.service ├─2690 /usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up └─2876 /usr/bin/docker-compose -f /usr/local/harbor/docker-compose.yml up Dec 01 11:43:33 node3 docker-compose[2690]: harbor-jobservice is up-to-date Dec 01 11:43:33 node3 docker-compose[2690]: nginx is up-to-date Dec 01 11:43:33 node3 docker-compose[2690]: Attaching to harbor-log, redis, harbor-adminserver, registry, harbor-db, harbor-ui, harbor-jobservice, nginx Dec 01 11:43:33 node3 docker-compose[2690]: harbor-adminserver | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: harbor-db | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: harbor-jobservice | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: harbor-ui | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: nginx | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: redis | WARNING: no logs are available with the 'syslog' log driver Dec 01 11:43:33 node3 docker-compose[2690]: registry | WARNING: no logs are available with the 'syslog' log driver
健康检查，如下。 都是healthy即可，如果是unhealthy，表示此镜像仓库不可用，一般这样的情况重启服务器即可恢复。
[root@node3 harbor]# docker-compose ps Name Command State Ports ------------------------------------------------------------------------------------------------------------------------------------- harbor-adminserver /harbor/start.sh Up (healthy) harbor-db /usr/local/bin/docker-entr ... Up (healthy) 3306/tcp harbor-jobservice /harbor/start.sh Up harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp harbor-ui /harbor/start.sh Up (healthy) nginx nginx -g daemon off; Up (healthy) 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp redis docker-entrypoint.sh redis ... Up 6379/tcp registry /entrypoint.sh serve /etc/ ... Up (healthy) 5000/tcp
登录私有镜像仓库，生成登录记录文件（注意，这个文件的生成不管是https的私有仓库还是http的仓库，只需要登录成功即可，有几个私有仓库，它都会记录进来的）：
登录记录文件是kubernetes使用私有镜像仓库的关键文件，一会会使用此文件生成一个secret，在使用私有仓库内的镜像时，将该secret挂载到部署文件内。
[root@node3 harbor]# docker login https://192.168.217.23 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded
[root@node3 harbor]# cat /root/.docker/config.json { "auths": { "192.168.217.23": { "auth": "YWRtaW46U2hpZ3VhbmdfMzI=" } }, "HttpHeaders": { "User-Agent": "Docker-Client/19.03.9 (linux)" }
三，
将密钥进行base64加密
cat /root/.docker/config.json | base64 -w 0
输出如下；
ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIxNy4yMyI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZVMmhwWjNWaGJtZGZNekk9IgoJCX0KCX0sCgkiSHR0cEhlYWRlcnMiOiB7CgkJIlVzZXItQWdlbnQiOiAiRG9ja2VyLUNsaWVudC8xOS4wMy45IChsaW51eCkiCgl9Cn0=
四，
新建secret部署文件，保存上述生成的密钥
cat >harbor_secret.yaml <<EOF apiVersion: v1 kind: Secret metadata: name: harbor-login type: kubernetes.io/dockerconfigjson data: # 这里添加上述base64加密后的密钥 .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIxNy4yMyI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZVMmhwWjNWaGJtZGZNekk9IgoJCX0KCX0sCgkiSHR0cEhlYWRlcnMiOiB7CgkJIlVzZXItQWdlbnQiOiAiRG9ja2VyLUNsaWVudC8xOS4wMy45IChsaW51eCkiCgl9Cn0= EOF
生成这个secret：
k apply -f harbor_secret.yaml
查看这个secret：
[root@node3 harbor]# k describe secrets harbor-login Name: harbor-login Namespace: default Labels: <none> Annotations: Type: kubernetes.io/dockerconfigjson Data ==== .dockerconfigjson: 152 bytes
五，
kubernetes的部署文件调用此secret
假设已有镜像上传到了私有镜像仓库内，该镜像是nginx，版本是1.20
现在部署nginx的时候指定使用私有镜像仓库里的镜像，那么，部署文件应该是这样的：
cat >nginx.yaml <<EOF apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: nginx name: nginx spec: replicas: 1 selector: matchLabels: app: nginx strategy: {} template: metadata: creationTimestamp: null labels: app: nginx spec: containers: - image: 192.168.217.23/library/nginx:1.20 name: nginx resources: {} imagePullSecrets: - name: harbor-login status: {} EOF
主要是在部署的时候增加了这么两行：
imagePullSecrets: - name: harbor-login
在部署一个MySQL，同样也是使用私有镜像仓库：
同样的，增加这么两行：
imagePullSecrets: - name: harbor-login
cat /etc/kubernetes/manifests/mysql.yaml apiVersion: v1 kind: Pod metadata: name: mysql labels: run: mysql name: mysql namespace: default spec: containers: - env: - name: MYSQL_ROOT_PASSWORD value: shiguang32 image: 192.168.217.23/test/mysql:5.7.39 imagePullPolicy: IfNotPresent resources: requests: cpu: 200m name: mysql volumeMounts: - mountPath: /var/lib/mysql name: mysql-data readOnly: false - mountPath: /etc/mysql/mysql.conf.d name: mysql-conf readOnly: false dnsPolicy: ClusterFirst restartPolicy: Always volumes: - name: mysql-data hostPath: path: /opt/mysql/data type: DirectoryOrCreate - name: mysql-conf hostPath: path: /opt/mysql/conf type: DirectoryOrCreate imagePullSecrets: - name: harbor-login hostNetwork: true priorityClassName: system-cluster-critical status: {}
[root@node3 harbor]# k get po NAME READY STATUS RESTARTS AGE mysql 1/1 Running 7 76m nginx-58bf645545-xtnsn 1/1 Running 1 161m
查看pod详情，看看是不是使用了私有仓库的镜像：
可以看到，确实是正确的下载了私有镜像仓库里的镜像
[root@node3 harbor]# k describe pod nginx-58bf645545-xtnsn Name: nginx-58bf645545-xtnsn Namespace: default Priority: 0 Node: node3/192.168.217.23 Start Time: Thu, 01 Dec 2022 10:59:50 +0800 Labels: app=nginx pod-template-hash=58bf645545 Annotations: <none> Status: Running IP: 10.244.0.243 IPs: IP: 10.244.0.243 Controlled By: ReplicaSet/nginx-58bf645545 Containers: nginx: Container ID: docker://07fc2a45709ff4698de6e4c168a175d1c10b9f23c1240c29fc1cb463142193c7 Image: 192.168.217.23/library/nginx:1.20 Image ID: docker-pullable://192.168.217.23/library/nginx@sha256:cba27ee29d62dfd6034994162e71c399b08a84b50ab25783eabce64b1907f774
[root@node3 harbor]# k describe pod mysql Name: mysql Namespace: default Priority: 2000000000 Priority Class Name: system-cluster-critical Node: node3/192.168.217.23 Start Time: Thu, 01 Dec 2022 12:24:50 +0800 Labels: run=mysql Annotations: Status: Running IP: 192.168.217.23 IPs: IP: 192.168.217.23 Containers: mysql: Container ID: docker://f6b6e9324bc17a2b3425edee382c01f0a3095379fbe4af4209bf4c7dc05bd55d Image: 192.168.217.23/test/mysql:5.7.39 Image ID: docker-pullable://192.168.217.23/test/mysql@sha256:b39b95329c868c3875ea6eb23c9a2a27168c3531f83c96c24324213f75793636
至此，kubernetes使用私有镜像仓库圆满成功！！！
