Is it possible to use gdb and qemu to debug linux user space programs and kernel space simultaneously?

Question

So far, with gdb + qemu, I can step into/over linux kernel source code. Is it possible to debug the user space programs simultaneously? For example, single step a program from user space to kernel space so I can observe the changes of registers on the qemu monitor by issuing info registers?

Answer 1

Minimal step-by-setep setup

Mahouk is right, but here is a fully automated QEMU + Buildroot example which presuposes that you already know how to debug the kernel with QEMU + gdb and a more detailed exaplanation:

readelf -h myexecutable | grep Entry

Gives:

  Entry point address:               0x4003a0

So inside GDB we need to do:

add-symbol-file myexecutable 0x4003a0
b main

And only then start the executable in QEMU:

myexecutable

A more reliable way to do that is to set myexecutable as the init process if you can do that.

add-symbol-file is also mentioned at: How to load multiple symbol files in gdb

Why would you ever want to do this instead of gdbserver?

I can only see one use case for this so far: debugging init: Debug init on Qemu using gdb

Otherwise, why not just use the following more reliable method, e.g. to step into a syscall:

• start two remote GDBs:
  • one with qemu-system-* -s
  • the other gdbserver myexecutable as explained at: https://reverseengineering.stackexchange.com/questions/8829/cross-debugging-for-mips-elf-with-qemu-toolchain/16214#16214
• step in gdbserver's GDB as close as possible to the system call, which often mean stepping into the libc
• on the QEMU's GDB, do e.g. b sys_read for the read syscall
• back on gdbserver, do continue

I propose this because:

• using the QEMU GDB for userland can lead to random jumps as the kernel context switches to another process that uses the same virtual addresses

• I was not able to load shared libraries properly without gdbserver: attempting sharedlibrary directly gives:

  (gdb) sharedlibrary ../../staging/lib/libc.so.0
  No loaded shared libraries match the pattern `../../staging/lib/libc.so.0'.
  

  As a consequence, since most kernel interactions go through the stdib, you would need to do a lot of smart assembly stepping to find the kernel entry, which could be impractical.

  Until, that is, someone writes a smarter GDB scripts that steps every instruction until a context switch happens or until source become available. I wonder if such scripts would't be too slow, as the naive approach has the overhead of communication to-from GDB for every instruction.

  This might get you started: Tell gdb to skip standard files

Parsing Linux kernel data structures

To do userland process debug properly, that's what we would have to do eventually: thread-aware gdb for the Linux kernel

Answer 2

I achieve it by using the gdb command add-symbol-file to add userspace programs debugging information. But you must know these programs loading addresses. so to be precise, you have to launch the kernel debugging by connecting gdb to gdbserver as usual; and then, you can add those program debugging information. You can also use .gdbinit script though. Read this

Answer 3

In mit xv6 os lab, we can use file command to switch symbol table between different executable including xv6 kernel or program running in user mode.

So I check whether it works or not on Linux kernel and its user programs. The result is that it also works, but we need to guarantee kernel and user programs are statically linked.

In summary, you can use the following steps to debug user programs:

1. When gdb and QEMU are connected, it is common that you have load vmlinux symbol table. Press ctrl+c to stop QEMU, the gdb is waiting for following command.
2. Use file [your user programs] command to switch symbol table.
3. Try set a breakpoint on your user programs , continue the gdb and running your user programs on QEMU.

Make sure that all programs are compiled with -g to build debug info and be statically linked.