0

Through some manipulation I have narrowed it down to a problem with the ret op. I know call pushes the return address to the stack; is it illegal to pop it and push it back?

format ELF64 executable 3

entry start

segment readable executable

start:
    pop rcx         ; argc
    mov [argc],cl       ; int -> ASCII
    add [argc],'0'
    push 1 argc 1
    call sys_write

    mov rdi,0
    mov rax,60
    syscall

sys_write: ; (fd,*buf,count)
    pop r11
    pop rdx rsi rdi
    mov rax,1
    syscall
    push r11
    ret

segment readable writable

argc rb 1

Output is:

$ ./prog
1Segmentation fault
$ _
Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
qrpnxz
  • 13
  • 5
  • 1
    SYSCALL destroys r11. You could have picked nearly any other register. But it's much easier to just pass function args in registers (like the ABI standard calling convention does), or to read the stack with `mov ..., [rsp+8]`. FASM's `pop rdx rsi rdi` syntax is just syntactical sugar, and it still assembles to 3 instructions. This isn't efficient. – Peter Cordes Oct 09 '16 at 20:57
  • If you'd used a debugger, you could have checked the register values. – Peter Cordes Oct 09 '16 at 20:59
  • You've got to be kidding me. Well, `r12` worked. Just my luck. Thank you very much. – qrpnxz Oct 09 '16 at 21:02
  • @PeterCordes Also, I know it's not efficient, but I wanted to write succinct syscalls for another, larger program I'm writing before having to write macros. Thanks for that `rsp` suggestion though. I've never thought of it. – qrpnxz Oct 09 '16 at 21:17

1 Answers1

1

The syscall instruction clobbers the contents of R11 with the RFLAGS register, which means that after making a syscall the value you stored in R11 is overwritten.

A solution might be to simply select an appropriate register that goes unmodified by a syscall. According to Intel's instruction reference manual (http://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-manual-325462.pdf, page 4-668 Vol. 2B) syscall overwrites RCX, RIP, R11, and RFLAGS as part of its operation, but the OS is of course able to restore RIP and RFLAGS.

This leaves plenty of other options. A good choice would be a register that a function call is allowed to clobber in the standard user-space calling convention, but that Linux system calls will leave unmodified. R8 fits the bill. Since you aren't using the standard x86-64 System V function-calling convention, RBX or RBP would be even better choices (smaller machine-code size since they don't need a REX prefix).

CL.
  • 173,858
  • 17
  • 217
  • 259
SevenStarConstellation
  • 108
  • 7
  • The `syscall` instruction appears to use `RCX` to store the return address from the syscall, so it doesn't seem to be available either. – SevenStarConstellation Oct 09 '16 at 21:03
  • /facepalm, yes of course. my mistake. I don't think there are any free call-clobbered low-8 registers for a 3-operand system call. (RBX and RBP are call-preserved). The best option is of course to not pop the return address in the first place. – Peter Cordes Oct 09 '16 at 21:06
  • Hang on, R8 isn't callee-saved in the [x86-64 SystemV ABI](http://stackoverflow.com/documentation/x86/3261/calling-conventions/11197/64-bit-system-v#t=201610092111176889144). And no, you never need to save/restore RSP or RFLAGS. Saving FLAGS is what SYSCALL uses R11 for (so it can be masked to enter the kernel with interrupts disabled or not, whatever the OS configures). The original values of RCX and R11 are destroyed, but everything else goes somewhere and is restored before or by the kernel's [SYSRET instruction](http://www.felixcloutier.com/x86/SYSRET.html). – Peter Cordes Oct 09 '16 at 21:10
  • Of course Linux system calls overwrite RAX with the return value, separately from the operation of the actually SYSCALL / SYSRET mechanism. – Peter Cordes Oct 09 '16 at 21:13
  • You're right, I was totally off about R8. RSP being restored looks like it depends on the OS syscall handler though, since `syscall` doesn't save it and `sysret` doesn't modify it on its own. – SevenStarConstellation Oct 09 '16 at 21:18
  • The SysV ABI specifies that all register other than RAX/RCX/R11 are unmodified by a system call, as explained in the linked duplicate question. Any sane OS will return to user-space with RSP unmodified, though. Letting the OS clobber other registers would be possible, but not RSP! Then user-space would need thread-local storage to save/restore RSP in a thread-safe manner... And yes, one of the first things Linux does in the SYSCALL entry point is PUSH most of the registers onto the kernel stack, to be restored later. I forget exactly how it saves user-RSP before switching to a kernel stack – Peter Cordes Oct 09 '16 at 21:33