Reset at jumping to Protected Mode

Question

I'm developing a bootloader to load my OS (I didn't use GRUB, because I wanted to learn assembler), and my code triple faults and resets QEMU. Here is the code in question:

Bootloader

 [ org 0x7c00 ]
 [ BITS 16 ]

jmp 0x0000:Start

%include 'PrintFunc.asm'
%include 'DiskOp.asm'

Start:
    cli
    xor ax, ax
    mov ss, ax
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax
    mov sp, 500h
    mov bp, 1500h
    cld
    sti
    
    mov ax, 0007h
    int 10h
    
    mov si, A1
    call printStr
    
    mov [BootDrive], dl
    
    mov al, 2
    
    call diskLoad
    
    mov si, A2
    call printStr
    
    jmp Cont
    
A1: db 'Loading sectors...', 0xA, 0xD, 0

BootDrive: db 0x00
    
times 510-($-$$) db 0
dw 0xAA55

A2: db 'Loaded two more sectors.', 0xA, 0xD, 0xA, 0
A3: db 'Checking A20...', 0xA, 0xD, 0
A4: db 'Enabling A20...', 0xA, 0xD, 0
A5: db 'A20 Enabled.', 0xA, 0xD, 0xA, 0
A6: db 'Loaded GDT, preparing to jump into PM.', 0xA, 0xD, 0
A7: db 'Landed in 32bit Protected Mode.', 0xA, 0xD, 0xA, 0
A8: db 0xA, 'Current FlameLoader version: ', 0
Ver: db '0.1', 0xA, 0xD, 0

GDT: 
    .NULL:
        dq 0
    
    .CodeSeg:
        dw 0FFFFh
        dw 0
        
        db 0
        db 010011010b
        db 011011111b
        db 0
    .DataSeg:
        dw 0FFFFh
        dw 0
        
        db 0
        db 010010010b
        db 011011111b
        db 0
    
    .end:
    
    .desc:
        dw .end - GDT - 1
        db GDT

%include 'A20Func.asm'
        
Cont:
    mov si, A3
    call printStr

    call testA20
    cmp ax, 1
    je EA20
    
    mov si, A4
    call printStr
    
    call enableA20
    
EA20:   ; A20 Enabled
    mov si, A5
    call printStr

    
    cli
    
    lgdt [GDT.desc]
    
    sti
        
    mov si, A6
    call printStr
    
    cli

    mov eax, cr0
    or eax, 1
    mov cr0, eax
    
    jmp 0x8:Init32

 [BITS 32]
    
Init32:
    
    jmp $   ; Debug
    
    mov ax, 0x10
    mov ds, ax
    mov ss, ax
    mov es, ax
    
    mov esp, 500h
    
    jmp Start32

%include 'Print32.asm'

Start32:
    
    mov esi, A7
    call PrintStr32
    
    jmp $   ; Actual program end

Based on BOCHS debugger and trying to stop the program at various points with jmp $, I have deduced that the problem lies with this line:

jmp 0x8:Init32

P.S. I didn't include the function files as I don't think they would be useful here.

Michael Petch · Accepted Answer · 2020-06-29T20:13:38.890

You don't show your disk load function but ultimately it would have to do something equivalent to:

    mov ax, 0201h
    mov cx, 0002h
    mov dh, 0
    mov bx, 7e00h
    int 13h

This would load CHS=(0,0,2) to ES:BX (0x0000:0x7e00) which is just after the bootloader. Alternatively you could have set ES:BX to 0x07e0:0x0000. Since you claim the problem seems the jump into protected mode I will assume the disk load is working.

That really only leaves the GDT that could be the problem other than a potential stack problem¹. I do see a significant problem here that could cause the FAR JMP into protected mode to fail:

.desc:
    dw .end - GDT - 1
    db GDT

The GDT Record (GDTR) should be a WORD (dw) with the length-1 which you have done followed by a DWORD (dd). You have defined the base as a single byte db! You need to change that to dd. It should look like:

.desc:
    dw .end - GDT - 1
    dd GDT

NASM unfortunately won't try to tell you a value it stuffed into a byte was narrowed (chopped) from another value. This issue likely resulted in your problems as the code descriptor would have been invalid causing the jmp 0x8:Init32 to triple fault and cause a reboot.

If debugging with BOCHS you could have set a breakpoint at 0x7c00 and stepped through instruction by instruction until the instruction after lgdt [GDT.desc]. You would have been able to view the GDT with info gdt. You would likely discover that the base is wrong and all the entries are incorrect.

Additional Notes

¹You have set the stack pointer SS:SP to 0x0000:0x0500. The stack grows down from that address. 0x0000:0x0500 contains data for the BIOS Data Area (BDA) and just below it is the real mode interrupt vector table (IVT). You should put the stack somewhere safer. 0x0000:0x7c00 growing down below the bootloader leaves a lot of stack room before it would clobber the BDA.
You need to turn off interrupts before entering protected mode so you could have simply done a CLI at the beginning of your code and avoided all the CLI/STI instructions. Alternatively you could have kept them enabled the entire time and just done a CLI just before entering protected mode.

It is often customary when doing things with interrupts enabled to update SS:SP together by first updating SS to a new value and SP. This is because updating SS has a side effect of disabling interrupts until after the following instruction. If updating SP is the following instruction then the operation is done atomically and no interrupt can occur between updating SS and SP. On some 8088 processors there was a defect where this didn't occur but that doesn't apply here since your code requires a 386+.
Setting BP in your real mode code has no effect since you aren't using stack frames anywhere and you aren't calling anything like a BIOS routine that requires BP to be set to a particular value. You also don't need to set GS and FS as your code isn't using them in real mode.

The start of your code could have appeared as:

Start:
    xor ax, ax
    mov ds, ax
    mov es, ax
    mov ss, ax     ; Set SS:SP to grow down beneath bootloader at 0x0000:0x7c00
    mov sp, 7c00h
    cld

I have addition Bootloader tips in this Stackoverflow answer that may be of use.

"The GDT Record (GDTR) should be a DWORD (`dw`) with the length-1 which you have done followed by a DWORD (`dd`)" -- The reference to `dw` should read **WORD**. — ecm, Jun 29 '20 at 20:11

Reset at jumping to Protected Mode

1 Answers1

Additional Notes