x86_64 assembly %rsp vs %esp

Question

I have been playing with assembly recently, and I came across a strange bug in my program. I have found that if I modify %rsp by doing 64-bit math, then everything works fine, but if I modify %esp by the same amount, except with 32-bit math, I get a segmentation fault. I tried printing out both %esp and %rsp, and they are the same every time I run.

QUESTION: Why does it matter whether I do 64-bit math or 32-bit math when the whole register is only using 32 bits?

.cstring
    _format: .asciz "%d\n"

.text
.globl _main
_main:
    # program setup
    pushq   %rbp
    movq    %rsp, %rbp

    # program - 16 byte aligned at this point
    # print stack pointer memory
    movq    %rsp, %rax
    call    bob            # prints the same value as the next call to bob
    xorq    %rax, %rax
    movl    %esp, %eax
    call    bob            # prints the same value as previous call to bob

    # this code breaks
    subl    $16, %esp      # bug here if I use this (32 bit math)
    subq    $16, %rsp      # works fine if I use this (64 bit math)
    call    bob
    addq    $16, %rsp

    # program cleanup
    movq    %rbp, %rsp
    popq    %rbp
    ret

# assumes 16 byte aligned when called. Prints %rax
 bob:
    subq    $8, %rsp
    movq    %rax, %rsi
    lea     _format(%rip), %rdi
    call    _printf
    addq    $8, %rsp
    ret

subl $16, %esp sets the upper 32 bits of RSP to zero. Kaboom. [Explained here](http://stackoverflow.com/a/25456097/17034). — Hans Passant, Jul 18 '15 at 01:53
Related: http://stackoverflow.com/questions/15656887/a-modification-to-esp-cause-sigsegv — nrz, Jul 19 '15 at 11:31

score 3 · Accepted Answer · answered Jul 18 '15 at 01:55

3

In x86_64, addresses are 64 bits, so how can you expect to do 32-bit math on it and still working fine? Moreover most 32-bit operations on x86_64 zero out the top 32 bits, render the address invalid

answered Jul 18 '15 at 01:55

phuclv

37,963
15
156
475

2

Not just most. Every instruction that uses a 32bit `%esomething` register as an output zeroed the upper 32. Thus, they don't depend on the previous value of the register, for out-of-order execution. – Peter Cordes Jul 18 '15 at 09:22

score 1 · Answer 2 · edited May 23 '17 at 11:58

1

With the help of conversation with KerrekSB, the reason for the discrepancy is a printing error. When I said I printed both RSP and ESP, I did so with a %d in a _printf call, which prints a 4 byte value. When applied to RSP, it only printed the lower 4 bytes (equivalent to ESP), leading me to believe that RSP only had meaningful data in the lower 4 bytes. When I changed it to print a %lu, I found that RSP and ESP are indeed not equivalent and RSP was using the upper 4 bytes. This was why 32 bit math caused a segmentation fault.

edited May 23 '17 at 11:58

Community

1
1

answered Jul 18 '15 at 01:56

Adam Evans

2,072
1
20
29

2

See http://stackoverflow.com/tags/x86/info for links to guides, and info on using gdb on asm code. This would have caught your error. – Peter Cordes Jul 18 '15 at 09:23

x86_64 assembly %rsp vs %esp

2 Answers2