kvm performance tuning for qemu-kvm daemon's high cpu usage-阿里云开发者社区

kvm performance tuning for qemu-kvm daemon's high cpu usage

2016-04-05 4078

版权

本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。

简介：

昨天写了两篇关于kvm虚拟机安装的文章

http://blog.163.com/digoal@126/blog/static/16387704020153142253971/

http://blog.163.com/digoal@126/blog/static/1638770402015315195563/

在装好后, 发现一个问题, 即使我的虚拟机上没跑任何用户进程, 在宿主机上对应的qemu-kvm进程的CPU利用率也有9.0左右.

删掉一些不必要的控制器(如USB), 然后添加CPU模块, 使用本地CPU的flag.

virsh # shutdown centos6_6_x64
virsh # edit centos6_6_x64
..................

原来的配置文件大概是这样的,

<domain type='kvm' id='2'>
  <name>centos6_6_x64</name>
  <uuid>4c613d4e-716b-f2cb-4df3-09bc7779f7df</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.6.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writethrough'/>
      <source file='/data02/kvmdisk/disk01.img'/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='block' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-0'/>
      <address type='drive' controller='0' bus='1' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb0'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb0'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb0'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x2'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:76:ac:2b'/>
      <source bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
    </input>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
</domain>

修改后的XML如下 :

virsh # dumpxml centos6_6_x64
<domain type='kvm'>
  <name>centos6_6_x64</name>
  <uuid>4c613d4e-716b-f2cb-4df3-09bc7779f7df</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.6.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writethrough'/>
      <source file='/data03/kvmdisk/disk01.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:76:ac:2b'/>
      <source bridge='virbr0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
</domain>

开机

virsh # start centos6_6_x64
Domain centos6_6_x64 started

virsh # console centos6_6_x64
Connected to domain centos6_6_x64
Escape character is ^]

查看当前的cpu flag, 判断为 Nehalem :

[root@db-172-16-3-150 ~]# ps -ewf|grep kvm
root       966     2  0 Apr01 ?        00:00:00 [kvm-irqfd-clean]
qemu     20418     1 31 10:18 ?        00:00:16 /usr/libexec/qemu-kvm -name centos6_6_x64 -S -M rhel6.6.0 -cpu Nehalem,+rdtscp,+dca,+pdcm,+xtpr,+tm2,+est,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -enable-kvm -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 4c613d4e-716b-f2cb-4df3-09bc7779f7df -nographic -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/centos6_6_x64.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/data03/kvmdisk/disk01.img,if=none,id=drive-virtio-disk0,format=raw,cache=writethrough -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=25,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:76:ac:2b,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on

现在qemu-kvm CPU使用率降低到0.x%

XML配置文件样本,

[root@db-172-16-3-150 ~]# cd /usr/share/libvirt/
[root@db-172-16-3-150 libvirt]# ll -R
.:
total 28K
-rw-r--r--. 1 root root  19K Jan 29 20:49 cpu_map.xml
drwxr-xr-x. 2 root root 4.0K Apr  1 13:33 networks
drwxr-xr-x. 2 root root 4.0K Feb 11 10:42 schemas

./networks:
total 4.0K
-rw-r--r--. 1 root root 230 Jan 29 20:49 default.xml

./schemas:
total 236K
-rw-r--r--. 1 root root 7.4K Jan 29 20:49 basictypes.rng
-rw-r--r--. 1 root root 8.7K Jan 29 20:49 capability.rng
-rw-r--r--. 1 root root 101K Jan 29 20:49 domaincommon.rng
-rw-r--r--. 1 root root  285 Jan 29 20:49 domain.rng
-rw-r--r--. 1 root root 3.9K Jan 29 20:49 domainsnapshot.rng
-rw-r--r--. 1 root root  12K Jan 29 20:49 interface.rng
-rw-r--r--. 1 root root 5.8K Jan 29 20:49 networkcommon.rng
-rw-r--r--. 1 root root  11K Jan 29 20:49 network.rng
-rw-r--r--. 1 root root 8.8K Jan 29 20:49 nodedev.rng
-rw-r--r--. 1 root root  30K Jan 29 20:49 nwfilter.rng
-rw-r--r--. 1 root root 1.6K Jan 29 20:49 secret.rng
-rw-r--r--. 1 root root  788 Jan 29 20:49 storageencryption.rng
-rw-r--r--. 1 root root  12K Jan 29 20:49 storagepool.rng
-rw-r--r--. 1 root root 4.9K Jan 29 20:49 storagevol.rng

还记得cluster.rng吗?

[root@db-172-16-3-150 libvirt]# cd /usr/share/cluster/
[root@db-172-16-3-150 cluster]# ll
total 20K
-rwxr-xr-x. 1 root root 2.4K Oct 15 19:45 checkquorum
-rw-r--r--. 1 root root 2.4K Oct 15 19:45 checkquorum.wdmd
lrwxrwxrwx. 1 root root   28 Feb 11 10:42 cluster.rng -> /var/lib/cluster/cluster.rng

[参考]

其他优化手段参考如下 :

1. http://www.linux-kvm.org/page/Tuning_KVM

2. http://wiki.qemu.org/Qemu-doc.html

3. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/Virtualization_Tuning_and_Optimization_Guide/index.html

4. /usr/share/libvirt