I am learning baremetal development on ARM, for which I chose to simulate Raspi3 on QEMU. Hence, its a virtual ARM Cortex A-53 imlementing ARMv8 architecture. I have compiled the following simple baremetal code :
.global _start
_start:
1: wfe
b 1b
I launch it using :
qemu-system-aarch64 -M raspi3 -kernel kernel8.img -display none -S -s
and the GDB is connected to it from the other terminal using :
gdb-multiarch ./kernel8.elf -ex 'target remote localhost:1234' -ex 'break *0x80000' -ex 'continue'
So far everything is good and I can notice the breakpoint in gdb.
Reading symbols from ./kernel8.elf...
Remote debugging using localhost:1234
0x0000000000000000 in ?? ()
Breakpoint 1 at 0x80000: file start.S, line 5.
Continuing.
Thread 1 hit Breakpoint 1, _start () at start.S:5
5 1: wfe
(gdb) info threads
Id Target Id Frame
* 1 Thread 1.1 (CPU#0 [running]) _start () at start.S:5
2 Thread 1.2 (CPU#1 [running]) 0x0000000000000300 in ?? ()
3 Thread 1.3 (CPU#2 [running]) 0x000000000000030c in ?? ()
4 Thread 1.4 (CPU#3 [running]) 0x000000000000030c in ?? ()
(gdb) list
1 .section ".text.boot"
2
3 .global _start
4 _start:
5 1: wfe
6 b 1b
(gdb)
As per my understanding, in case of ARM all the cores will execute the same code on reset, so ideally all the cores in my case must be running the same code. I just want to verify that by putting breakpoints and that is the problem. The break points for other cores are not hit. If I am not wrong, the threads in my case are nothing but the cores. I tried putting break but does not work :
(gdb) break *0x80000 thread 2
Note: breakpoint 1 (all threads) also set at pc 0x80000.
Breakpoint 2 at 0x80000: file start.S, line 5.
(gdb) thread 2
[Switching to thread 2 (Thread 1.2)]
#0 0x0000000000000300 in ?? ()
(gdb) info threads
Id Target Id Frame
1 Thread 1.1 (CPU#0 [running]) _start () at start.S:5
* 2 Thread 1.2 (CPU#1 [running]) 0x0000000000000300 in ?? ()
3 Thread 1.3 (CPU#2 [running]) 0x000000000000030c in ?? ()
4 Thread 1.4 (CPU#3 [running]) 0x000000000000030c in ?? ()
(gdb) s
Cannot find bounds of current function
(gdb) c
Continuing.
[Switching to Thread 1.1]
Thread 1 hit Breakpoint 1, _start () at start.S:5
5 1: wfe
(gdb)
I deleted the core 1 breakpoint, and then the core 2 hangs forever :
(gdb) info b
Num Type Disp Enb Address What
1 breakpoint keep y 0x0000000000080000 start.S:5
breakpoint already hit 2 times
2 breakpoint keep y 0x0000000000080000 start.S:5 thread 2
stop only in thread 2
(gdb) delete br 1
(gdb) info break
Num Type Disp Enb Address What
2 breakpoint keep y 0x0000000000080000 start.S:5 thread 2
stop only in thread 2
(gdb) thread 2
[Switching to thread 2 (Thread 1.2)]
#0 0x000000000000030c in ?? ()
(gdb) c
Continuing.
What can I do get a breakpoint on core 2? What am I doing wrong here?
EDIT
I tried set scheduler-locking on (assuming this is what I need) but this also seems not working for me.
(gdb) break *0x80000
Breakpoint 3 at 0x80000: file start.S, line 5.
(gdb) thread 2
[Switching to thread 2 (Thread 1.2)]
#0 0x000000000000030c in ?? ()
(gdb) set scheduler-locking on
(gdb) c
Continuing.
^C/build/gdb-OxeNvS/gdb-9.2/gdb/inline-frame.c:367: internal-error: void skip_inline_frames(thread_info*, bpstat): Assertion `find_inline_frame_state (thread) == NULL' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) n
This is a bug, please report it. For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.
/build/gdb-OxeNvS/gdb-9.2/gdb/inline-frame.c:367: internal-error: void skip_inline_frames(thread_info*, bpstat): Assertion `find_inline_frame_state (thread) == NULL' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n)
EDIT 2
Upon @Frank's advice, I built (latest) qemu 6.2.0 locally and used the gdb available in the arm toolchain.
naveen@workstation:~/.repos/src/arm64/baremetal/raspi3-tutorial/01_bareminimum$ /opt/qemu-6.2.0/build/qemu-system-aarch64 -version
QEMU emulator version 6.2.0
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
naveen@workstation:~/.repos/src/arm64/baremetal/raspi3-tutorial/01_bareminimum$ /opt/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gdb -version
GNU gdb (GNU Toolchain for the A-profile Architecture 10.3-2021.07 (arm-10.29)) 10.2.90.20210621-git
But I am still having the problem. My other cores 2,3 and 4 never hit the breakpoints. It seems they are not even running my code, as the address they are pointing to, does not look ok.
(gdb) info threads
Id Target Id Frame
* 1 Thread 1.1 (CPU#0 [running]) _start () at start.S:5
2 Thread 1.2 (CPU#1 [running]) 0x000000000000030c in ?? ()
3 Thread 1.3 (CPU#2 [running]) 0x000000000000030c in ?? ()
4 Thread 1.4 (CPU#3 [running]) 0x000000000000030c in ?? ()
EDIT 3
The problem seems with my Makefile, as when I used the command to build, as suggested by Frank, it worked for me. Can someone please look as what's wrong with this Makefile :
CC = /opt/gcc-arm-10.3-2021.07-x86_64-aarch64-none-elf/bin/aarch64-none-elf
CFLAGS = -Wall -O2 -ffreestanding -nostdinc -nostartfiles -nostdlib -g
all: clean kernel8.img
start.o: start.S
${CC}-gcc $(CFLAGS) -c start.S -o start.o
kernel8.img: start.o
${CC}-ld -g -nostdlib start.o -T link.ld -o kernel8.elf
${CC}-objcopy -O binary kernel8.elf kernel8.img
clean:
rm kernel8.elf kernel8.img *.o >/dev/null 2>/dev/null || true
EDIT 4
It turns out that when I use kernel8.elf with QEMU for booting, everything works as expected. But when I use kernel8.img which is a binary format, I get the issue. With bit of reading, I understand that ELF contains the "extra" information required to make the example work. But for clarification, how can I make the kernel8.img work?
