作者：牧原
第一个问题
limits.conf的限制在/proc/pid/limits中未生效
# cat /proc/3606/limits
Limit Soft Limit Hard Limit Units
Max processes 31202 31202 processes
Max open files 1024 4096 files在Centos7 & ubuntu 系统中，使用Systemd替代了之前的SysV。/etc/security/limits.conf文件的配置作用域缩小了。
/etc/security/limits.conf的配置，只适用于通过PAM认证登录用户的资源限制，它对systemd的service的资源限制不生效。因此登录用户的限制，通过/etc/security/limits.conf与/etc/security/limits.d下的文件设置即可。
对于systemd service的资源设置，则需修改全局配置，
全局配置文件放在/etc/systemd/system.conf和/etc/systemd/user.conf，
同时也会加载两个对应目录中的所有.conf文件/etc/systemd/system.conf.d/.conf和/etc/systemd/user.conf.d/.conf
system.conf是系统实例使用的，user.conf是用户实例使用的。
vim /etc/systemd/system.conf
DefaultLimitNOFILE=100000
DefaultLimitNPROC=65535修改并重启即可
# cat /proc/3613/limits
Limit Soft Limit Hard Limit Units
Max processes 65535 65535 processes
Max open files 100000 100000 files第二个问题
在服务里面设置LimitNOFILE=infinity为什么不是无穷大？
在服务里面设置LimitNOFILE=infinity 后，通过查看pid的limit发现openfile是65536 ，而不是无穷大
查看服务配置
[root@iZwz98aynkjcxvtra0f375Z ~]# cat /etc/systemd/system/multi-user.target.wants/docker.service |grep -vi "^#"|grep -vi "^$"
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd://
ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target查看配置效果
# cat /proc/11019/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 31202 31202 processes
Max open files 65536 65536 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 31202 31202 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us这个是systemd的bug，低于240的版本需要手动设置才可以生效
LimitNOFILE=102400
https://github.com/systemd/systemd/issues/6559
第三个问题
为什么openfile不能设置为unlimited
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n
65535
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n unlimited
-bash: ulimit: open files: cannot modify limit: Operation not permitted原因是 centos7里 openfile不能大于nr_open
[root@iZwz98aynkjcxvtra0f375Z ~]# cat /proc/sys/fs/nr_open
1048576
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n 1048577
-bash: ulimit: open files: cannot modify limit: Operation not permitted
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n 1048576
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n
1048576第四个问题
使用supervisor管理进程（测试环境ubuntu 1604）启动进程后，maxfile是1024
需要修改配置文件
#cat /etc/supervisor/supervisord.conf
[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)
下面这两行
minfds=655350 ; min. avail startup file descriptors; default 1024
minprocs=65535 ; min. avail process descriptors;default 200
# cat /proc/2423/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 65535 65535 processes
Max open files 655350 655350 files ------------修改成功
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 61946 61946 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us关于file-max nr_open file_nr的解释 可参考
https://www.kernel.org/doc/Documentation/sysctl/fs.txt
外部文档
https://www.cnblogs.com/zengkefu/p/5635153.html
https://blog.csdn.net/google0802/article/details/52304776
http://blog.cloud.360.cn/post/tuning-your-system-for-high-concurrency.html
https://blog.csdn.net/qq_38165374/article/details/104881340
