本文将测试一下使用cgroup的cpuset组来控制进程对CPU的使用 :
测试环境CentOS 7.x x64
创建一个继承组
[root@150 rg1]# cd /sys/fs/cgroup/cpuset/
[root@150 cpuset]# mkdir cg1
继承组自动创建对应的限制文件
[root@localhost cpuset]# cd cg1
[root@localhost cg1]# ll
total 0
-rw-r--r-- 1 root root 0 Jan 8 00:46 cgroup.clone_children
-rw-r--r-- 1 root root 0 Jan 8 00:46 cgroup.procs
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.cpu_exclusive
-rw-r--r-- 1 root root 0 Jan 8 03:06 cpuset.cpus
-r--r--r-- 1 root root 0 Jan 8 00:46 cpuset.effective_cpus
-r--r--r-- 1 root root 0 Jan 8 00:46 cpuset.effective_mems
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.mem_exclusive
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.mem_hardwall
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.memory_migrate
-r--r--r-- 1 root root 0 Jan 8 00:46 cpuset.memory_pressure
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.memory_spread_page
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.memory_spread_slab
-rw-r--r-- 1 root root 0 Jan 8 03:01 cpuset.mems
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.sched_load_balance
-rw-r--r-- 1 root root 0 Jan 8 00:46 cpuset.sched_relax_domain_level
-rw-r--r-- 1 root root 0 Jan 8 00:46 notify_on_release
-rw-r--r-- 1 root root 0 Jan 8 03:01 tasks
继承组的tasks为空.
查看CPU资源
设置cpuset.mems
# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 26
Model name: Intel(R) Xeon(R) CPU E5504 @ 2.00GHz
Stepping: 5
CPU MHz: 1995.053
BogoMIPS: 3989.39
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 4096K
NUMA node0 CPU(s): 0-7设置cpuset.mems
[root@localhost cg1]# echo 0 > ./cpuset.mems
否则在将PID添加到这个组的时候, 会报错如下 :
设置cpuset.cpus
[root@localhost cg1]# echo 48413 >./tasks
-bash: echo: write error: No space left on device
设置cpuset.cpus
[root@localhost cg1]# echo 7 > ./cpuset.cpus
将数据库的主进程加入到这个组
[root@localhost cg1]# ps -ewf|grep postgres
postgres 48413 1 0 Jan07 ? 00:00:00 /opt/pgsql9.3.5/bin/postgres
[root@localhost cg1]# echo 48413 >./tasks
现在数据库被限制了只能使用7号CPU 核.
使用top的P选项查看 :
postgres@localhost-> pgbench -M prepared -n -r -f ./test.sql -c 16 -j 4 -h 127.0.0.1 -T 30
transaction type: Custom query
scaling factor: 1
query mode: prepared
number of clients: 16
number of threads: 4
duration: 30 s
number of transactions actually processed: 560871
tps = 18695.122321 (including connections establishing)
tps = 18746.566935 (excluding connections establishing)
statement latencies in milliseconds:
0.851678 select 1;使用top的P选项查看 :
在top界面按
f
选择
查看到所有postgres进程新fork的backend process都用的7号cpu
* P = Last Used Cpu (SMP) 查看到所有postgres进程新fork的backend process都用的7号cpu
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND P
54349 postgres 20 0 307264 10076 8308 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14172) idle 7
54351 postgres 20 0 307264 10076 8308 S 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14174) idle 7
54353 postgres 20 0 307264 10140 8372 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14176) idle 7
54354 postgres 20 0 307264 10076 8308 R 7.3 0.0 0:00.80 postgres: postgres postgres 127.0.0.1(14177) idle 7
54355 postgres 20 0 307264 9900 8136 S 7.3 0.0 0:00.80 postgres: postgres postgres 127.0.0.1(14178) idle 7
54356 postgres 20 0 307264 9836 8068 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14179) idle 7
54357 postgres 20 0 307268 10080 8308 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14180) idle 7
54358 postgres 20 0 307268 10016 8244 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14181) idle 7
54360 postgres 20 0 307268 9956 8184 R 7.3 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14183) idle 7
54361 postgres 20 0 307268 10020 8248 R 7.3 0.0 0:00.78 postgres: postgres postgres 127.0.0.1(14184) idle 7
54350 postgres 20 0 307264 10076 8308 R 3.7 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14173) idle 7
54352 postgres 20 0 307264 10076 8308 R 3.7 0.0 0:00.79 postgres: postgres postgres 127.0.0.1(14175) idle 7
54359 postgres 20 0 307268 10020 8248 R 3.7 0.0 0:00.78 postgres: postgres postgres 127.0.0.1(14182) idle 7
54362 postgres 20 0 307268 9956 8184 R 3.7 0.0 0:00.78 postgres: postgres postgres 127.0.0.1(14185) idle 7
54363 postgres 20 0 307268 9960 8188 R 3.7 0.0 0:00.78 postgres: postgres postgres 127.0.0.1(14186) idle 7
54364 postgres 20 0 307268 9956 8184 R 3.7 0.0 0:00.77 postgres: postgres postgres 127.0.0.1(14187) idle 7
48413 postgres 20 0 306188 15896 15036 S 0.0 0.0 0:00.28 /opt/pgsql9.3.5/bin/postgres 7
改为0-7可以使用8个CPU核
[root@localhost cg1]# echo 0-7 > ./cpuset.cpus
再次测试 :
当然性能也有提升
[参考]
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
54424 postgres 20 0 307264 10004 8236 S 35.6 0.0 0:04.28 postgres: postgres postgres 127.0.0.1(14191) BIND 4
54423 postgres 20 0 307264 10076 8308 S 35.1 0.0 0:04.23 postgres: postgres postgres 127.0.0.1(14190) idle 4
54425 postgres 20 0 307264 10076 8308 R 34.6 0.0 0:04.24 postgres: postgres postgres 127.0.0.1(14192) idle 3
54433 postgres 20 0 307268 9336 7580 S 34.0 0.0 0:03.97 postgres: postgres postgres 127.0.0.1(14200) idle 6
54426 postgres 20 0 307264 10116 8348 S 33.5 0.0 0:04.20 postgres: postgres postgres 127.0.0.1(14193) idle 3
54427 postgres 20 0 307264 10076 8308 S 33.5 0.0 0:03.99 postgres: postgres postgres 127.0.0.1(14194) idle 6
54428 postgres 20 0 307268 9652 7892 R 33.0 0.0 0:04.05 postgres: postgres postgres 127.0.0.1(14195) idle 6
54429 postgres 20 0 307268 9652 7892 R 32.5 0.0 0:04.06 postgres: postgres postgres 127.0.0.1(14196) idle 1
54434 postgres 20 0 307268 10020 8248 S 31.9 0.0 0:03.92 postgres: postgres postgres 127.0.0.1(14201) idle 5
54437 postgres 20 0 307268 9896 8124 R 31.9 0.0 0:03.91 postgres: postgres postgres 127.0.0.1(14204) idle 2
54438 postgres 20 0 307268 9956 8184 R 31.9 0.0 0:03.85 postgres: postgres postgres 127.0.0.1(14205) BIND 0
54430 postgres 20 0 307268 9648 7888 R 31.4 0.0 0:03.99 postgres: postgres postgres 127.0.0.1(14197) idle 3
54431 postgres 20 0 307268 9940 8168 S 31.4 0.0 0:03.91 postgres: postgres postgres 127.0.0.1(14198) idle 3
54432 postgres 20 0 307268 9340 7588 S 31.4 0.0 0:03.92 postgres: postgres postgres 127.0.0.1(14199) idle 4
54435 postgres 20 0 307268 9956 8184 S 30.9 0.0 0:03.80 postgres: postgres postgres 127.0.0.1(14202) idle 7
54436 postgres 20 0 307268 9956 8184 R 30.9 0.0 0:03.85 postgres: postgres postgres 127.0.0.1(14203) BIND 7 当然性能也有提升
postgres@localhost-> pgbench -M prepared -n -r -f ./test.sql -c 16 -j 4 -h 127.0.0.1 -T 30
transaction type: Custom query
scaling factor: 1
query mode: prepared
number of clients: 16
number of threads: 4
duration: 30 s
number of transactions actually processed: 2924931
tps = 97490.141264 (including connections establishing)
tps = 97533.454952 (excluding connections establishing)
statement latencies in milliseconds:
0.162975 select 1;
