# 导入加载镜像,test.tar大小有16G左右 docker load -i test.tar
报错信息
制作镜像失败:【ApplyLayer exit status1 stdout:stderr:write /home/appuser/.local/MIb/python3.7/site- packages/scipy/linalg/tests/pycache_/test_decomp_update.cpython-37.pyc:no space left on device
问题排查
是Base Device Size的问题,其默认值是10G,而容器经过一段时间的运行写入文件量早已超过10G,就造成无法写入数据的问题了
Doker 的默认配置
# docker info [root@fly ~]# docker info Containers: 8 Running: 8 Paused: 0 Stopped: 0 Images: 8 Server Version: 18.09.0 Storage Driver: devicemapper Pool Name: docker-0:108-164868621-pool Pool Blocksize: 65.54kB Base Device Size: 10.74GB Backing Filesystem: xfs Udev Sync Supported: true Data file: /dev/loop0 Metadata file: /dev/loop1 Data loop file: /public/docker/devicemapper/devicemapper/data Metadata loop file: /public/docker/devicemapper/devicemapper/metadata Data Space Used: 846.6MB Data Space Total: 107.4GB Data Space Available: 106.5GB Metadata Space Used: 19.81MB Metadata Space Total: 17.05GB Metadata Space Available: 17.03GB Thin Pool Minimum Free Space: 10.74GB Deferred Removal Enabled: true Deferred Deletion Enabled: true Deferred Deleted Device Count: 0 Library Version: 1.02.149-RHEL7 (2018-07-20) Logging Driver: json-file Cgroup Driver: systemd Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: c4446665cb9c30056f4998ed953e6d4ff22c7c39 runc version: 4fc53a81fb7c994640722ac585fa9ca548971871 init version: fec3683 Security Options: seccomp Profile: default Kernel Version: 3.10.0-957.el7.x86_64 Operating System: CentOS Linux 7 (Core) OSType: linux Architecture: x86_64 CPUs: 32 Total Memory: 188.2GiB Name: admin1 ID: 5IMT:GNNU:MFJS:IJJ6:RZJS:7CA7:N5QC:ZSF6:223N:5CB6:NFFM:W2N6 Docker Root Dir: /public/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 192.168.172.128:5000 192.168.172.128:5001 127.0.0.0/8 Registry Mirrors: http://hub-mirror.c.163.com/ https://registry.docker-cn.com/ https://docker.mirrors.ustc.edu.cn/ Live Restore Enabled: false Product License: Community Engine WARNING: bridge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
- Base Device Size: 10.74GB
- Data Space Total: 107.4GB
- Metadata Space Total: 17.05GB
问题解决
调整docker服务端启动参数
# vim /usr/lib/systemd/system/docker.service [root@fly ~]# grep "ExecStart=" /usr/lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -s=devicemapper --storage-opt dm.basesize=60G --storage-opt dm.loopmetadatasize=20G
dm.basesize
默认为10G,限制容器和镜像的大小dm.loopdatasize
存储池大小,默认为100Gdm.loopmetadatasize
元数据大小,默认为2Gdm.datadev
存储池设备,默认生成一个/var/lib/docker/devicemapper/devicemapper/data
文件dm.metadatadev
元数据设备,默认生成一个/var/lib/docker/devicemapper/devicemapper/metadata
文件dm.fs
用于基本映像的文件系统(xfs或ext4)dm.blocksize
:精简池的自定义块大小。默认值为64K。
具体可参考官方文档: https://docs.docker.com/engine/reference/commandline/dockerd/
当一个容器的数据空间大于10GB后,那么这个容器将不能写入新的数据文件。如果容器需要很大的数据空间,可以使用数据卷挂在到宿主机或存储上。当容器太多使用数据卷挂载方式后,所有容器使用的数据空间已经大于100GB,这时将不能新建或运行容器。如何突破这种限制,这篇文章给出了解答:https://github.com/docker/docker/tree/master/daemon/graphdriver/devmapper
分两种情况讨论:
1、如果docker第一次启动,在启动docker前,可以使用参数–storage-opt分别指定dm.basesize、dm.loopdatasize、dm.loopmetadatasize等项,指定单个容器可用数据空间、docker可用数据空间、元数据可用数据空间。
2、如果是正在运行的docker,需要导出镜像docker commit、docker push、docker save等操作,然后停止docker服务,删除docker的数据文件(centos6.6是rm -rf /var/lib/docker/)。再修改docker的配置文件/etc/sysconfig/docker,重启docker服务生效。最后导入备份的镜像启动容器。
修改完配置后, 重启docker
systemctl daemon-reload systemctl restart docker
查看调整
# ps -ef | grep docker|grep /usr/bin/dockerd|grep -v grep [root@fly ~]# ps -ef | grep docker|grep /usr/bin/dockerd|grep -v grep root 23665 1 9 10月12 ? 01:25:52 /usr/bin/dockerd --bip=173.0.52.1/24 --ip-masq=true --mtu=1450 --graph /public/docker --storage-opt dm.basesize =60G --storage-opt dm.loopmetadatasize=20G
参考
https://docs.docker.com/engine/reference/commandline/dockerd/
https://github.com/moby/moby/tree/master/daemon/graphdriver/devmapper