How to download the Torvalds Linux Kernel master, (re)compile it, and boot it with QEMU?

Question

Since few days, I am working on a little project in order to discover the kernel programming. However, I read a lot on the internet and asked a question before about kernel compilation and I feel still lost at some point.

If I need to do some kernel programming, I thought that use the Linus Torvalds's Linux sources would be the best point to start. So I did the following (From MacOS High Sierra):

• Download kernel's sources: https://github.com/torvalds/linux
• Unzipped it
• Launch a terminal and run:
  • vagrant init
  • vagrant up
  • vagrant ssh
• Once connected, I typed:
  • cd /vagrant/linux In order to get into the Linux sources unzipped folder
  • make menuconfig I just pressed ok/save
  • sudo make -j 4 && sudo make modules_install -j 4 && sudo make install -j 4

Now, I just get a little error:

agrant @vagrant-ubuntu-trusty-64:/vagrant/Kernel-Programming/linx-kernel$ sudo make -j 4 && sudo make modules_install -j 4 && sudo make install -j 4
  CHK include/config/kernel.release
CHK     include/generated/uapi/linux/version.h
CHK     include/generated/utsrelease.h
CHK     scripts/mod/devicetable-offsets.h
CHK     include/generated/timeconst.h
CHK     include/generated/bounds.h
CHK     include/generated/asm-offsets.h
CALL    scripts/checksyscalls.sh
CHK     include/generated/compile.h
SKIPPED include/generated/compile.h
CC      my_syscalls/uptime.o
AR      my_syscalls/built-in.o
CC[M]  net/ipv4/netfilter/ipt_ECN.o
In file included from net/ipv4/netfilter/ipt_ECN.c:20:0:
net/ipv4/netfilter/ipt_ECN.c: In function 'set_ect_ip':
./include/uapi/linux/netfilter_ipv4/ipt_ECN.h:16:27: error: 'XT_DSCP_MASK' undeclared(first use in this function)
 #define IPT_ECN_IP_MASK (~XT_DSCP_MASK)
                           ^
net/ipv4/netfilter/ipt_ECN.c:33:18: note: in expansion of macro 'IPT_ECN_IP_MASK'
  if ((iph->tos & IPT_ECN_IP_MASK) != (einfo->ip_ect & IPT_ECN_IP_MASK)) {
                  ^
./include/uapi/linux/netfilter_ipv4/ipt_ECN.h:16:27: note: each undeclared identifier is reported only once for each function it appears in
 #define IPT_ECN_IP_MASK (~XT_DSCP_MASK)
                           ^
net/ipv4/netfilter/ipt_ECN.c:33:18: note: in expansion of macro 'IPT_ECN_IP_MASK'
  if ((iph->tos & IPT_ECN_IP_MASK) != (einfo->ip_ect & IPT_ECN_IP_MASK)) {
                  ^
net/ipv4/netfilter/ipt_ECN.c: In function 'ecn_tg_check':
./include/uapi/linux/netfilter_ipv4/ipt_ECN.h:16:27: error: 'XT_DSCP_MASK' undeclared(first use in this function)
 #define IPT_ECN_IP_MASK (~XT_DSCP_MASK)
                           ^
net/ipv4/netfilter/ipt_ECN.c:105:23: note: in expansion of macro 'IPT_ECN_IP_MASK'
  if (einfo->ip_ect & ~IPT_ECN_IP_MASK)
{
                       ^
make[3]: *** [net / ipv4 / netfilter / ipt_ECN.o] Error 1
make[2]: *** [net / ipv4 / netfilter] Error 2
make[1]: *** [net / ipv4] Error 2
make: *** [net] Error 2
make: ***Waiting for unfinished jobs....

However, I don't know if this error breaks everything at all or not. Also, even if the compilation is fully ok, how can I test my "new Linux compiled"? Because I added a syscall and I would like to test if it works or no ... I found something about update-initramfs but I don't see the link/point since my compiled kernel isn't an Ubuntu.

Answer 1

Buildroot

https://buildroot.org/

Buildroot is an awesome way to do this.

In very few commands it:

• downloads GCC and generates the cross compiler
• downloads the Linux kernel, ulibc etc. and generates a root image
• downloads QEMU and compiles it for you

For aarch64 the commands are:

git clone https://github.com/buildroot/buildroot
cd buildroot
git checkout 2018.02
make qemu_aarch64_virt_defconfig
printf '
BR2_CCACHE=y
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE=n
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
BR2_PACKAGE_HOST_QEMU_VDE2=y
' >>.config
make olddefconfig
time make BR2_JLEVEL="$(nproc)" HOST_QEMU_OPTS='--enable-sdl --with-sdlabi=2.0'
./output/host/usr/bin/qemu-system-aarch64 \
  -M virt \
  -cpu cortex-a57 \
  -nographic \
  -smp 1 \
  -kernel output/images/Image \
  -append "root=/dev/vda console=ttyAMA0" \
  -netdev user,id=eth0 \
  -device virtio-net-device,netdev=eth0 \
  -drive file=output/images/rootfs.ext4,if=none,format=raw,id=hd0 \
  -device virtio-blk-device,drive=hd0 \
;

You are now left on a shell and you can login with username root (no password).

The QEMU boot command is documented in Buildroot itself at: https://github.com/buildroot/buildroot/blob/2018.02/board/qemu/aarch64-virt/readme.txt

Then to use your own Linux kernel source, you basically just have to use LINUX_OVERRIDE_SRCDIR as explained at: How to modify the source of Buildroot packages for package development?

And then basically for free you also get a way to:

• cross compile userland tests to see what your kernel is doing by creating a hello world package like this
• cross compile kernel modules out of tree, which is a good way to interact with the kernel directly without touching the mainline tree: How to add a Linux kernel driver module as a Buildroot package?

I have automated and documented all of those things on this GitHub repo. To make kernel development even better, that:

• repo compiles the kernel directly rather than going through Buildroot
• provides shortcuts to run and GDB step debug the kernel

Other ISAs mentioned at: https://cirosantilli.com/linux-kernel-module-cheat/#buildroot-hello-world

Tested in Ubuntu 18.04.

Answer 2

Good work please document the work and post it on git / blog lot of tweaking required to get linux kernel compile on mac OS X.

Now to answer your question.

1. Easiest way to run vmware/virtualbox. as they are straight forward. then scp image and test.
2. QEMU is best way to test your kernel. but not widely used by general population. Also needs bit of efforts to get it working. Also need to compile Root FS to boot as you want to test syscall.(you can write c code using your syscall and define it as init process)

I hope I have answered you.