2

I am having an issue on some servers since some time, and fail to find the issue. These are x86_64 server, with Intel Xeon, configured to boot in UEFI over network, through an iPXE rom. Kernel and initramfs are the ones from Centos 8 (tried 8.0 and 8.2).

But when booting, I always end up with (on every servers, so should not be related to an hardware failure):

[    5.542304] hid-generic 0003:0557:2221.0002: input,hidraw1: USB HID v1.00 Keyboard [Winbond Electronics Corp Hermon USB hidmouse Device] on usb-0000:00:1a.0-1.3/input1
[    5.599611] rtc_cmos 00:02: setting system clock to 2020-06-26 19:21:43 UTC (1593199303)
[    5.620965] md: Waiting for all devices to be available before autodetect
[    5.640580] md: If you don't use raid, use raid=noautodetect
[    5.659949] md: Autodetecting RAID arrays.
[    5.676869] md: autorun ...
[    5.691838] md: ... autorun DONE.
[    5.707883] List of all partitions:
[    5.723667] No filesystem could mount root, tried:
[    5.723667]
[    5.754724] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    5.775237] CPU: 9 PID: 1 Comm: swapper/0 Not tainted 4.18.0-193.6.3.el8_2.x86_64 #1
[    5.815778] Call Trace:
[    5.830360]  dump_stack+0x5c/0x80
[    5.845920]  panic+0xe7/0x2a9
[    5.860995]  mount_block_root+0x2c5/0x2e9
[    5.877407]  ? do_early_param+0x91/0x91
[    5.892617]  prepare_namespace+0x135/0x16b
[    5.907676]  kernel_init_freeable+0x22e/0x258
[    5.922607]  ? rest_init+0xaa/0xaa
[    5.937398]  kernel_init+0xa/0xff
[    5.950991]  ret_from_fork+0x35/0x40
[    5.964572] Kernel Offset: 0x2ca00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    6.000387] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---

The iPXE boot script is:

#!ipxe

kernel http://10.10.0.1/vmlinuz-4.18.0-193.6.3.el8_2.x86_64 initrd=initramfs-4.18.0-193.6.3.el8_2.x86_64.img selinux=0  rd.shell rd.debug root=live:http://10.10.0.1/squashfs.img rw console=tty0 console=ttyS1,115200
initrd http://10.10.0.1/initramfs-4.18.0-193.6.3.el8_2.x86_64.img

boot

Which generate this in the console:

http://10.10.0.1/vmlinuz-4.18.0-193.6.3.el8_2.x86_64... ok
http://10.10.0.1/initramfs-4.18.0-193.6.3.el8_2.x86_64.img... ok
INTEL 0x6f080f70 MAC reset (081c0261/80280783 was 081c0261/80280783)
INTEL 0x6f080f70 MAC reset (081c0261/80280783 was 081c0261/80280783)
INTEL 0x6f081ab0 MAC reset (081c0261/80280787 was 081c0261/80280787)
[    0.000000] Linux version 4.18.0-193.6.3.el8_2.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)) #1 SMP Wed Jun 10 11:09:32 UTC 2020
[    0.000000] Command line: vmlinuz-4.18.0-193.6.3.el8_2.x86_64 initrd=initramfs-4.18.0-193.6.3.el8_2.x86_64.img selinux=0 rd.shell rd.debug root=live:http://10.10.0.1/squashfs.img rw console=tty0 console=ttyS1,115200
...

On the server side, on apache logs I have:

10.10.2.1 - - [26/Jun/2020:19:55:42 +0200] "GET /vmlinuz-4.18.0-193.6.3.el8_2.x86_64 HTTP/1.1" 200 8913656 "-" "iPXE/1.0.0+"
10.10.2.1 - - [26/Jun/2020:19:55:42 +0200] "GET /initramfs-4.18.0-193.6.3.el8_2.x86_64.img HTTP/1.1" 200 53703611 "-" "iPXE/1.0.0+"

So it seems to be working perfectly.

This is diskless boot here, but whatever I try (kickstart diskfull install, or even kernel+initrd alone) I always end up to this kernel panic...

I tried to reset BIOS settings, try to boot in legacy/pcbios instead of UEFI, tried to desactivate sata disks, etc. Always this same error. Also tried to use kernel+initrd from Centos ISO (checked checksum), tried to use the ones from my management node. Nothing. Do I miss something obvious?

Does any of you have an idea or already faced this kind of issue?

Many thanks in advance :-)

With my best regards

Beuk

beuk
  • 21
  • 1
  • 4
  • Have you tried to drop `initrd=...` parameter from the kernel command line? I think it tries to find file while it should look for address in memory where PXE prepared the file. Also you may try to supply `rootfstype=ramfs`. – 0andriy Jun 26 '20 at 23:04
  • Thank you for this proposal. I tried it, no better result. So I tried other distributions kernel/initramfs, same. Then tried to revert to "old way", i.e. PXE using syslinux based this time one slow tftp instead of http, it worked, but failed later when downloading packages. This means that my http server is probably broken (strangely, link is stable during normal ssh/operations, but when on high load, it drops). I tried with another server as http server and it worked. I still need to understand why iPXE said the initrd was 100% downloaded while I am pretty sure it got an empty file. – beuk Jun 27 '20 at 20:10
  • for PXE you should just `APPEND initramfs.img,rootfs.img` although i have the same issue with iPXE – TecHunter Oct 11 '20 at 12:52
  • The kernel shows that it can't find/use the initrd. in EFI mode you MUST provide the initrd argument to the kernel command line so that the kernel knows what to load from "efifs" (`iPXE` does not do this for you, while `PXELINUX` might), to me your root seems weird, did you try `root=/dev/ram0`. To debug http issues, did you tcpdump? – NiKiZe Sep 08 '21 at 21:00

0 Answers0