How does one Capture the Entire Kernel Panic on Boot

Question

Using Buildroot, I'm attempting to make a custom kernel build. After building the image and booting it on a VirtualBox environment, the kernel always panics after the GRUB stage. General summary of what I see:

] CPU: 0 PID: 1 ...
] Hardware name: innotek GmbH ...
] <some registers>
] Call Trace:
]  [<c0a1c995>] dump_stack+...
]  [<........>] panic+...
]  [<........>] do_exit+...
]  ...
] Kernel Offset: 0x0 from 0xc0400000 ...
] ---[ end Kernel panic - not syncing: Attempted to kill init! ...

Now, I'm assuming this is only the tail of the message that I want to see, but I have no means to view it (ex: cannot Shift-PageUp). When the panic occurs, the above text is never rendered for a second on the screen.

I first stumbled upon a KernelDebuggingTricks page which states:

Slowing down kernel messages on boot

... [Build] the kernel with the following option enabled:

CONFIG_BOOT_PRINTK_DELAY=y

And boot the machine with the following kernel boot parameter:

boot_delay=N

I have confirmed that my kernel (3.16) had built with the CONFIG_BOOT_PRINTK_DELAY option and have tried setting boot_delay to 10, 500 and 1000 milliseconds in my GRUB. Even with the 1000 millisecond delay set (and waiting for a good ~5 minutes), the entire kernal panic log message is spewed out in a blink of an eye.

Does anyone have any suggestions on how I can view the root of the kernel panic? The only thing I'm thinking now is manually adding sleeps into the kernel code (which is something I'd like to avoid).

*"the above text is never rendered for a second on the screen."* -- Huh? Can you state what happens in positive terms and without the negation? Have you tried assigning the system console to a serial port, aka tty, (in the kernel command line) which would allow logging/capture? — sawdust, Aug 15 '14 at 21:46
Tried the serial port as yourself and duskwulf mentioned; thanks. — jdknight, Aug 18 '14 at 14:26
any idea, what was the issue ? I am also trying to load custom kernel in VM environment and its panicking similar to yours ... any tips how you proceeded ? — app, Dec 07 '16 at 18:30

score 8 · Accepted Answer · answered Aug 15 '14 at 21:47

8

Configure your virtual machine to log to the serial console as well as the console by adding console=ttyS0 console=tty0 to the boot parameters, then configure the serial port of your machine (in the VirtualBox settings) to output to a file. If a kernel panic occurs, the details will end up in that file.

For details, see: https://www.virtualbox.org/wiki/Serial_redirect

answered Aug 15 '14 at 21:47

Thanks! Exactly what I was looking for. Used the serial port with the Port Mode `Raw File`. – jdknight Aug 18 '14 at 14:25

score 0 · Answer 2 · answered Jul 05 '17 at 20:35

kdump and netdump are two more options if you don't have the convenience of a serial port, which is the case when running on metal for most current laptops. See this answer for more info: https://unix.stackexchange.com/a/60928/32558 The setup is more involved however.

Finally, you might also be interested in step debugging the kernel with JTAG or QEMU (not sure about Virtualbox support): Linux kernel live debugging, how it's done and what tools are used?

How does one Capture the Entire Kernel Panic on Boot

2 Answers2