qemu + gdb: The efficient way to understand/debug Linux kernel code/data structure
2 likes823 views
The document provides a detailed guide for setting up and debugging the Linux kernel using qemu and gdb in an x86_64 environment, primarily based on kernel 5.11. It includes instructions for environment preparation, launching the virtual machine, and specific gdb commands for examining kernel data structures. Examples of breakpoints and memory structures are also discussed to aid in understanding the debug process.
![Page Map
Level-4 Table
40
CR3 init_top_pgt = swapper_pg_dir
Sign-extend
Page Map
Level-4 Offset Physical Page Offset
0
30 21
39 20
38 29
47
48
63
Page Directory
Pointer Offset
Page Directory
Offset
Page Directory
Pointer Table
Page Directory
Table
level3_kernel_pgt
PDPTE #511
PDPTE #510 PDE #506
PDE #507
PDE #505
Direct Mapping Region
Kernel Code & fixmap
cpu_entry_area: 0.5TB
vmalloc: 32TB
PDE #13
PML4E #402
PML4E #273
…
PML4E #465
PML4E #468
PML4E #508
PML4E #511
vmemmap (page
descriptor)
PDPTE #0
Page Table Offset
1211
PTE #82 = 0
PTE #83 = 0
Page Table
Physical Memory
page frame
Example #4: Page Table Examination
[Linear Address] 0xffff_c900_01a5_2000, 0xffff_c900_01a5_3000
1
2
3
4
5](https://image.slidesharecdn.com/qemugdb-220826072647-500ee596/85/qemu-gdb-The-efficient-way-to-understand-debug-Linux-kernel-code-data-structure-12-320.jpg)
qemu + gdb: The efficient way to understand/debug Linux kernel code/data structure
- 1. * Based on kernel 5.11 (x86_64) – QEMU * 2-socket CPUs (4 cores/socket) * 16GB memory * Kernel parameter: nokaslr norandmaps * KASAN: disabled * Userspace: ASLR is disabled * Host OS: Ubuntu 20.04.1 qemu + gdb: The efficient way to understand/debug Linux kernel code/data structure Adrian Huang | Aug, 2022
- 2. Environment Preparation • Repo: https://github.com/AdrianHuang/gdb-linux-real-mode.git
- 3. Environment Preparation • Repo: https://github.com/AdrianHuang/gdb-linux-real-mode.git o ./scripts/build.sh ▪ Download/build kernel and busybox: Make sure if your machine can access the Internet o ./scripts/launch-vm.sh ▪ Launch a guest OS (QEMU) and wait for gdb connection o ./scripts/launch-gdb.sh ▪ Launch gdb debugger and connect to the QEMU guest OS • Steps: $ git clone https://github.com/AdrianHuang/gdb-linux-real-mode.git Cloning into 'gdb-linux-real-mode’... $ cd gdb-linux-real-mode/ $ ./scripts/build.sh # After build.sh is done, you’re all set!
- 4. QEMU/gdb at a glance: Console #1 Console #1: wait for gdb debugger
- 5. QEMU/gdb at a glance: Console #2: Real-mode entry point Console #2: gdb debugger – breakpoint @0x10200 (real-mode entry point) Kernel boot section 0x10000 0x10200 Physical Memory QEMU loader loads ‘setup.bin’ at address 0x10000 0 ds = es = fs = gs = ss cs stack sp = 0x1FFF0 (ss:0xFFF0) protected mode real mode Kernel setup code gdb command file: Add any gdb commands in this file Reference (Real-mode entry point): Vmlinux: anatomy of bzimage and how x86 64 processor is booted
- 6. Console #2: step & continue 1 2 Console #1: QEMU: Guest OS 3 QEMU/gdb at a glance: Console #1/#2 Reference (Real-mode entry point): Vmlinux: anatomy of bzimage and how x86 64 processor is booted
- 7. Note: Debug the decompressed vmlinux (generic kernel) $ head -n 12 gdb-files/gdb-linux-kernel-real-mode.txt # debug info about real-mode code of Linux kernel add-symbol-file /home/adrian/work/gdb-linux-real-mode/out/obj/linux/arch/x86/boot/setup.elf 0x103ff -s .bstext 0x10000 -s .bsdata 0x1002d -s .header 0x101ef -s .entrytext 0x1026c -s .inittext 0x102d4 -s .initdata 0x103e1 -s .text32 0x130ce -s .bss 0x136e0 -s .data 0x13660 # debug info about compressed vmlinux add-symbol-file /home/adrian/work/gdb-linux-real-mode/out/obj/linux/arch/x86/boot/compressed/vmlinux 0x3ce4f0 - s .head.text 0x100000 -s .data 0x3d5b90 -s .bss 0x3d5e40 -s .pgtable 0x3f6000 target remote :1234 # Uncomment the following line if you want to debug the decompressed vmlinux add-symbol-file /home/adrian/work/gdb-linux-real-mode/out/obj/linux/vmlinux set print pretty on
- 8. Kernel supported gdb functions and commands
- 9. Console #2: gdb console Console #1: QEMU: Guest OS Example #1: Conditional breakpoint gdb-files/gdb-linux-kernel-real-mode.txt
- 10. Example #2: watchpoint Only one pte mapped pte: physical address 1 2 3 gdb command: (gdb) watch *(0xffff8881809c0028)
- 11. 4KB Disk Mapping Layer: file system Generic Block Layer sector size bi_size = 1024 bvec_iter bi_sector bv_len = 1024 bio_vec bv_page bv_offset bio Example #3: Data structure examination
- 12. Page Map Level-4 Table 40 CR3 init_top_pgt = swapper_pg_dir Sign-extend Page Map Level-4 Offset Physical Page Offset 0 30 21 39 20 38 29 47 48 63 Page Directory Pointer Offset Page Directory Offset Page Directory Pointer Table Page Directory Table level3_kernel_pgt PDPTE #511 PDPTE #510 PDE #506 PDE #507 PDE #505 Direct Mapping Region Kernel Code & fixmap cpu_entry_area: 0.5TB vmalloc: 32TB PDE #13 PML4E #402 PML4E #273 … PML4E #465 PML4E #468 PML4E #508 PML4E #511 vmemmap (page descriptor) PDPTE #0 Page Table Offset 1211 PTE #82 = 0 PTE #83 = 0 Page Table Physical Memory page frame Example #4: Page Table Examination [Linear Address] 0xffff_c900_01a5_2000, 0xffff_c900_01a5_3000 1 2 3 4 5
- 13. Example #5: ptype: print data type