场景
如果虚拟机没有开启kdump,或者卡死了,那么可以进入qemu的monitor模式将虚机的内存保存到文件中,然后使用crash工具进行分析。
示例
- 启动虚拟机,然后按
ctrl+a c
进入monitor,查看帮助
(qemu) help dump-guest-memory dump-guest-memory [-p] [-d] [-z|-l|-s|-w] filename [begin length] -- dump guest memory into file 'filename'. -p: do paging to get guest's memory mapping. -d: return immediately (do not wait for completion). -z: dump in kdump-compressed format, with zlib compression. -l: dump in kdump-compressed format, with lzo compression. -s: dump in kdump-compressed format, with snappy compression. -w: dump in Windows crashdump format (can be used instead of ELF-dump converting), for Windows x64 guests with vmcoreinfo driver only. begin: the starting physical address. length: the memory size, in bytes.
- 保存虚机的全部内存,采用zlib压缩
(qemu) dump-guest-memory -z guest.img $ ls -lh guest.img -rwxrwxrwx 1 root root 241M Sep 22 05:17 guest.img
- 使用crash工具进行分析
$ crash guest.img linux-6.2/vmlinux crash 8.0.3 Copyright (C) 2002-2022 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011, 2020-2022 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. Copyright (C) 2015, 2021 VMware, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 10.2 Copyright (C) 2021 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... please wait... (determining panic task) KERNEL: linux-6.2/vmlinux DUMPFILE: guest.img [PARTIAL DUMP] CPUS: 8 DATE: Fri Sep 22 05:16:53 PDT 2023 UPTIME: 00:01:12 LOAD AVERAGE: 0.39, 0.14, 0.05 TASKS: 181 NODENAME: ubuntu-vm RELEASE: 6.2.0+ VERSION: #4 SMP PREEMPT_DYNAMIC Mon Aug 14 02:28:01 PDT 2023 MACHINE: x86_64 (3194 Mhz) MEMORY: 8 GB PANIC: "" PID: 0 COMMAND: "swapper/0" TASK: ffffffff8362cac0 (1 of 8) [THREAD_INFO: ffffffff8362cac0] CPU: 0 STATE: TASK_RUNNING (ACTIVE) WARNING: panic task not found crash>
查看dmesg:
crash> dmesg [ 0.000000] Linux version 6.2.0+ (pengdl@ubuntu) (gcc (Ubuntu 8.4.0-3ubuntu2) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #4 SMP PREEMPT_DYNAMIC Mon Aug 14 02:28:01 PDT 2023 [ 0.000000] Command line: noinitrd rootfstype=ext4 root=/dev/vda rw console=ttyS0 crashkernel=1G nokaslr [ 0.000000] x86/fpu: x87 FPU will use FXSAVE [ 0.000000] signal: max sigframe size: 1440 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffdcfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffdd000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000004] kvm-clock: using sched offset of 1180034822 cycles [ 0.000009] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000040] tsc: Detected 3194.004 MHz processor [ 0.001718] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved ...