1、Containerd镜像管理
1.1 Containerd容器镜像管理命令
- docker使用
docker images命令管理镜像 - 单机containerd使用
ctr images命令管理镜像,containerd本身的CLI - k8s中containerd使用
crictl images命令管理镜像,Kubernetes社区的专用CLI工具
获取命令帮助 # ctr --help NAME: ctr - __ _____/ /______ / ___/ __/ ___/ / /__/ /_/ / \___/\__/_/ ......
获取命令帮助 # ctr images NAME: ctr images - manage images USAGE: ctr images command [command options] [arguments...] COMMANDS: check check existing images to ensure all content is available locally export export images import import images list, ls list images known to containerd mount mount an image to a target path unmount unmount the image from the target pull pull an image from a remote push push an image to a remote delete, del, remove, rm remove one or more images by reference tag tag an image label set and clear labels for an image convert convert an image OPTIONS: --help, -h show help
1.2 查看镜像
# ctr images ls REF TYPE DIGEST SIZE PLATFORMS LABELS
1.3 下载镜像
containerd支持oci标准的镜像，所以可以直接使用docker官方或dockerfile构建的镜像
# 镜像名不能简写，--all-platforms：所有平台，不加的话下载当前平台架构 # ctr images pull --all-platforms docker.io/library/nginx:alpine # ctr images pull docker.io/library/nginx:latest # uname -a
说明： 这里ctr命令pull镜像时，不能直接把镜像名字写成`nginx:alpine`
查看已下载容器镜像 # ctr images ls
| REF | TYPE | DIGEST |
| docker.io/library/nginx:alpine | application/vnd.docker.distribution.manifest.list.v2+json | sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 |
| SIZE | PLATFORMS | LABELS |
| 9.7 MiB | linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x | - |
指定平台下载容器镜像 # ctr images pull --platform linux/amd64 docker.io/library/nginx:alpine
没有镜像是无法运行容器的。
1.4 镜像挂载
方便查看镜像中包含的内容。
把已下载的容器镜像挂载至当前文件系统 # ctr images mount docker.io/library/nginx:alpine /mnt sha256:af2fcce448e2e4451a5f4796a9bf9cb5c9b5f88e0d6d10029cada42fb9d268ac /mnt [root@localhost ~]# ls /mnt bin dev docker-entrypoint.d docker-entrypoint.sh etc home lib media mnt opt proc root run sbin srv sys tmp usr var
卸载 # umount /mnt
1.5 镜像导出
把容器镜像导出 # ctr i export --all-platforms nginx.img docker.io/library/nginx:alpine
说明 --all-platforms,导出所有平台镜像，本版本为1.6版本，1.4版本不需要添加此选项。
查看已导出容器镜像 # ls nginx.img # ls -lh 总用量 196M -rw-r--r-- 1 root root 73M 2月 18 14:48 nginx.img
1.6 镜像删除
删除指定容器镜像 # ctr image rm docker.io/library/nginx:alpine docker.io/library/nginx:alpine 再次查看容器镜像 [root@192 ~]# ctr images ls REF TYPE DIGEST SIZE PLATFORMS LABELS
1.7 镜像导入
导入容器镜像 # ctr images import nginx.img unpacking docker.io/library/nginx:alpine (sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3)...done
1.8 修改镜像tag
# ctr images tag docker.io/library/nginx:alpine nginx:alpine nginx:alpine # 建议修改tag的名称为：镜像仓库地址/目录/镜像名
说明： 把docker.io/library/nginx:alpine 修改为 nginx:alpine
查看修改后的容器镜像 # ctr images ls REF TYPE DIGEST SIZE PLATFORMS LABELS docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x - nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
修改后对容器镜像做检查比对 # ctr images check REF TYPE DIGEST STATUS SIZE UNPACKED docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true
2、Containerd容器管理
2.1 获取命令帮助
2.1.1 获取ctr命令帮助
[root@localhost ~]# ctr --help NAME: ctr - __ _____/ /______ / ___/ __/ ___/ / /__/ /_/ / \___/\__/_/ containerd CLI ......
2.1.2 获取创建静态容器命令帮助
# ctr container --help NAME: ctr containers - manage containers USAGE: ctr containers command [command options] [arguments...] COMMANDS: create create container delete, del, remove, rm delete one or more existing containers info get info about a container list, ls list containers label set and clear labels for a container checkpoint checkpoint a container restore restore a container from checkpoint OPTIONS: --help, -h show help
说明： 使用`ctr container create `命令创建容器后，容器并没有处于运行状态，其只是一个静态的容器。这个 container 对象只是包含了运行一个容器所需的资源及配置的数据结构，例如： namespaces、rootfs 和容器的配置都已经初始化成功了，只是用户进程(本案例为nginx)还没有启动。需要使用`ctr tasks`命令才能获取一个动态容器。
2.1.3 获取动态容器命令帮助
# ctr run --help NAME: ctr run - run a container ......
说明： 使用`ctr run`命令可以创建一个静态容器并使其运行。一步到位运行容器。
2.2 查看容器
container表示静态容器，可用c缩写代表container
# ctr container ls CONTAINER IMAGE RUNTIME
或
# ctr c ls CONTAINER IMAGE RUNTIME
以上命令无法查看出来容器的静态的还是动态的
2.3 查看任务
task表示容器里跑的进程, 可用t缩写代表task
# ctr task ls TASK PID STATUS
或
# ctr t ls TASK PID STATUS
2.4 创建静态容器
# ctr c create docker.io/library/nginx:alpine nginx
# ctr container ls CONTAINER IMAGE RUNTIME nginx1 docker.io/library/nginx:alpine io.containerd.runc.v2
查看容器详细信息 # ctr container info nginx1
2.5 静态容器启动为动态容器
复制containerd连接runC垫片工具至系统 # ls usr/local/bin/ containerd containerd-shim containerd-shim-runc-v1 containerd-shim-runc-v2 containerd-stress crictl critest ctd-decoder ctr [root@localhost ~]# cp usr/local/bin/containerd-shim-runc-v2 /usr/bin/
启动task，即表时在容器中运行了进程，即为动态容器。 # ctr task start -d nginx1 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明： -d表示daemon或者后台的意思，否则会卡住终端
查看容器所在宿主机进程，是以宿主机进程的方式存在的。 # ctr task ls TASK PID STATUS nginx1 3395 RUNNING
查看容器的进程(都是物理机的进程) # ctr task ps nginx1 PID INFO 3395 - 3434 -
物理机查看到相应的进程 # ps -ef | grep 3395 root 3395 3375 0 19:16 ? 00:00:00 nginx: master process nginx -g daemon off; 101 3434 3395 0 19:16 ? 00:00:00 nginx: worker process
2.6 进入容器操作
# ctr task exec --exec-id 1 nginx1 /bin/sh # ifconfig 查看网卡信息 ... # curl 127.0.0.1 访问本地提供的web服务 ...
说明： 为exec进程设定一个id，可以随意输入，只要保证唯一即可，也可使用$RANDOM变量。
2.7 直接运行一个动态容器
# ctr run -d --net-host docker.io/library/nginx:alpine nginx2 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明： * -d 代表dameon，后台运行 * --net-host 代表容器的IP就是宿主机的IP(相当于docker里的host类型网络)
查看已运行容器 # ctr container ls CONTAINER IMAGE RUNTIME nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
查看已运行容器中运行的进程,既tasks # ctr tasks ls TASK PID STATUS nginx2 4061 RUNNING
进入容器 # ctr task exec --exec-id 1 -t nginx2 /bin/sh
/ # ifconfig ens33 Link encap:Ethernet HWaddr 00:0C:29:B1:B6:1D inet addr:192.168.10.164 Bcast:192.168.10.255 Mask:255.255.255.0 inet6 addr: fe80::2b33:40ed:9311:8812/64 Scope:Link inet6 addr: fe80::adf4:a8bc:a1c:a9f7/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:55360 errors:0 dropped:0 overruns:0 frame:0 TX packets:30526 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:53511295 (51.0 MiB) TX bytes:2735050 (2.6 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:68 errors:0 dropped:0 overruns:0 frame:0 TX packets:68 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:5916 (5.7 KiB) TX bytes:5916 (5.7 KiB) virbr0 Link encap:Ethernet HWaddr 52:54:00:E9:51:82 inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0 UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
为容器中运行的网站添加网站文件 / # echo "nginx2" > /usr/share/nginx/html/index.html / # exit
在宿主机上访问网站 [root@localhost ~]# curl 192.168.10.164 nginx2
2.8 暂停容器
查看容器状态 # ctr tasks ls TASK PID STATUS nginx2 4061 RUNNING
暂停容器 # ctr tasks pause nginx2
再次查看容器状态，看到其状态为PAUSED，表示停止。 # ctr tasks ls TASK PID STATUS nginx2 4061 PAUSED
[root@localhost ~]# curl http://192.168.10.164 在宿主机访问，发现不可以访问到网站
2.9 恢复容器
使用resume命令恢复容器 # ctr tasks resume nginx2
查看恢复后状态 # ctr tasks ls TASK PID STATUS nginx2 4061 RUNNING
在宿主机上访问容器中提供的服务 # curl http://192.168.10.164 nginx2
2.10 停止容器
# ctr tasks --help ......
使用kill命令停止容器中运行的进程，既为停止容器 # ctr tasks kill nginx
查看容器停止后状态，STATUS为STOPPED # ctr tasks ls TASK PID STATUS nginx1 3395 RUNNING nginx2 4061 STOPPED
2.11 删除容器
# ctr tasks delete nginx2 必须先删除task，再删除容器
查看静态容器，确认其还存在于系统中 # ctr container ls CONTAINER IMAGE RUNTIME nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
删除容器 # ctr container delete nginx2
3、Containerd使用私有容器镜像仓库 Harbor
3.1 Harbor准备
3.2 配置Containerd使用Harbor仓库
3.2.1 Harbor主机名解析
在所有安装containerd宿主机上添加此配置信息。
# vim /etc/hosts # cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.10.165 harbor.kubemsb.com
说明 * 192.168.10.165是harbor的IP * harbor.kubemsb.com建议用FQDN形式，如果用类似harbor这种短名，后面下载镜像会出问题
3.2.2 修改Containerd配置文件
此配置文件已提前替换过，仅修改本地容器镜像仓库地址即可。 # vim /etc/containerd/config.toml # cat /etc/containerd/config.toml root = "/var/lib/containerd" state = "/run/containerd" oom_score = -999 [grpc] address = "/run/containerd/containerd.sock" uid = 0 gid = 0 max_recv_message_size = 16777216 max_send_message_size = 16777216 [debug] address = "" uid = 0 gid = 0 level = "" [metrics] address = "" grpc_histogram = false [cgroup] path = "" [plugins] [plugins.cgroups] no_prometheus = false [plugins.cri] stream_server_address = "127.0.0.1" stream_server_port = "0" enable_selinux = false sandbox_image = "easzlab/pause-amd64:3.2" stats_collect_period = 10 systemd_cgroup = false enable_tls_streaming = false max_container_log_line_size = 16384 [plugins.cri.containerd] snapshotter = "overlayfs" no_pivot = false [plugins.cri.containerd.default_runtime] runtime_type = "io.containerd.runtime.v1.linux" runtime_engine = "" runtime_root = "" [plugins.cri.containerd.untrusted_workload_runtime] runtime_type = "" runtime_engine = "" runtime_root = "" [plugins.cri.cni] bin_dir = "/opt/kube/bin" conf_dir = "/etc/cni/net.d" conf_template = "/etc/cni/net.d/10-default.conf" [plugins.cri.registry] [plugins.cri.registry.mirrors] [plugins.cri.registry.mirrors."docker.io"] endpoint = [ "https://docker.mirrors.ustc.edu.cn", "http://hub-mirror.c.163.com" ] [plugins.cri.registry.mirrors."gcr.io"] endpoint = [ "https://gcr.mirrors.ustc.edu.cn" ] [plugins.cri.registry.mirrors."k8s.gcr.io"] endpoint = [ "https://gcr.mirrors.ustc.edu.cn/google-containers/" ] [plugins.cri.registry.mirrors."quay.io"] endpoint = [ "https://quay.mirrors.ustc.edu.cn" ] [plugins.cri.registry.mirrors."harbor.kubemsb.com"] 在此处添加,在镜像加速器下面添加这一段 endpoint = [ "http://harbor.kubemsb.com" ] [plugins.cri.x509_key_pair_streaming] tls_cert_file = "" tls_key_file = "" [plugins.diff-service] default = ["walking"] [plugins.linux] shim = "containerd-shim" runtime = "runc" runtime_root = "" no_shim = false shim_debug = false [plugins.opt] path = "/opt/containerd" [plugins.restart] interval = "10s" [plugins.scheduler] pause_threshold = 0.02 deletion_threshold = 0 mutation_threshold = 100 schedule_delay = "0s" startup_delay = "100ms"
重启containerd，以便于重新加载配置文件。 # systemctl daemon-reload # systemctl restart containerd
3.2.3 ctr下载镜像
下载容器镜像 # ctr images pull --platform linux/amd64 docker.io/library/nginx:latest
说明: * --platform linux/amd64 指定系统平台，也可以使用--all-platforms指定所有平台镜像。
查看已下载容器镜像 # ctr images ls REF TYPE DIGEST SIZE PLATFORMS LABELS docker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
3.2.4 ctr上传镜像
上传到Harbor library公有项目
重新生成新的tag # ctr images tag docker.io/library/nginx:latest harbor.kubemsb.com/library/nginx:latest harbor.kubemsb.com/library/nginx:latest
查看已生成容器镜像 # ctr images ls REF TYPE DIGEST SIZE PLATFORMS LABELS docker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x - harbor.kubemsb.com/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
推送容器镜像至Harbor # ctr images push --platform linux/amd64 --plain-http -u admin:密码 harbor.kubemsb.com/library/nginx:latest
说明: * 先tag再push * 因为我们harbor是http协议，不是https协议，所以需要加上`--plain-http` * `--user admin:Harbor12345`指定harbor的用户名与密码
输出： manifest-sha256:0fd68ec4b64b8dbb2bef1f1a5de9d47b658afd3635dc9c45bf0cbeac46e72101: done |++++++++++++++++++++++++++++++++++++++| config-sha256:dd025cdfe837e1c6395365870a491cf16bae668218edb07d85c626928a60e478: done |++++++++++++++++++++++++++++++++++++++| elapsed: 0.5 s total: 9.3 Ki (18.1 KiB/s)
下载已上传容器镜像 # ctr images pull --plain-http harbor.kubemsb.com/library/nginx:latest
