0

I have written a simple, basic swap function in asm in one file. In another file I have written a 'so-called' main program in order to test my swap function. However, as soon as the function is called, a segmentation fault occurs. I have no clue why this is happening. 

global swap

;given two addresses of stack variables (rdi, rsi) swaps the content of the two
;variables
swap:
enter 0, 0


mov rax, [rdi] ;save of var1
mov rdx, [rsi] ;save of var2
mov rdi, rbx
mov rsi, rax 

leave
ret

The function receives the addresses of the two stack variables to swap in rdi and rsi; here is the code to test the function:

global _start
extern swap

section .text
_start:
enter 16, 0
mov qword [rbp], 0xFF
mov qword [rbp+8], 0x11

mov rdi, [rbp]
mov rsi, [rbp+8]
call swap  ;swap causes seg fault?? ------------------

    cmp qword [rbp], 0xFF
    jz FAIL
    cmp qword [rbp+8], 0x11
    jz FAIL
    cmp qword [rbp], 0x11
    jnz FAIL
    cmp qword [rbp+8], 0xFF
    jnz FAIL

;else success

mov rax, 60
xor rdi, rdi
syscall

 FAIL:
    mov rax, 60
    mov rdi, -1
    syscall

The segmentation fault occurs when I call the swap function. I suspect it has to do with accessing the stack of the main program, but I do not know how else this could be done. I also have a make file compiling and linking the files together (No problem there as far as I know)

A.D
  • 427
  • 1
  • 4
  • 12
  • 1
    "If the callee wishes to use registers **RBX**, RBP, and R12–R15, **it must restore their original values before returning** control to the caller. " ([source](https://en.wikipedia.org/wiki/X86_calling_conventions#System_V_AMD64_ABI)) – Michael May 10 '19 at 11:16
  • 1
    Also, you didn't pass the addresses of the values to `swap` - you passed the actual values. If you wanted the addresses, use `lea` instead of `mov`. – Michael May 10 '19 at 11:18
  • @Michael doesn't the enter instruction do this automatically? Also, I didn't think that these caller and callee registers we anything more than best practices; they can actually cause errors? – A.D May 10 '19 at 11:19
  • @Michael The swap function was supposed to swap the contents of the two addresses, I believe it is correct in that sense – A.D May 10 '19 at 11:20
  • 1
    `enter` only saves `rbp`. And no, in this particular case it probably doesn't matter that you don't follow the ABI, but you might want to do that anyway. – Michael May 10 '19 at 11:21
  • 1
    No, `enter 0,0` is just a very slow way to write `push rbp` / `mov rbp, rsp`. Never use it if you care about performance. (`leave` is fine though). And no, it doesn't preserve RBX. But that's not the actual bug. Anyway, **use a debugger** to see what register values you're actually passing to the function, and that they're values not addresses. – Peter Cordes May 10 '19 at 11:21
  • _"The swap function was supposed to swap the contents of the two addresses"_ But you're not giving it any addresses. You're giving it the values at those addresses. – Michael May 10 '19 at 11:22
  • @Michael Ah yes, I'll edit that; seg fault is still there though... – A.D May 10 '19 at 11:24
  • Of course it is, you're passing `0xFF` as a pointer like Michael said a couple times. And a much better fix would be to use `rdx` or `rcx` as your 2nd tmp reg, not to save/restore RBX. Saving RBX is irrelevant anyway unless you want to call this from C; your asm caller doesn't care about RBX. This should all be obvious if you single-step with a debugger and look at the instruction that actually faults. – Peter Cordes May 10 '19 at 11:26
  • @PeterCordes "you're passing 0xFF as a pointer" could you elaborate on this, quit new to assembly and I'm not sure what you mean – A.D May 10 '19 at 11:27
  • 2
    What you're doing is like calling it from C as `swap( (void*)a, (void*)b)` instead of `swap(&a, &b)`. **Use a debugger to look at register values.** I assume you know what each instruction does individually, so when you get to `mov rax, [rdi]` with RDI=0xff, it should be obvious why that faults. – Peter Cordes May 10 '19 at 11:29
  • @PeterCordes beginers mistake... Thanks for spotting that, everything works now :) – A.D May 10 '19 at 11:33

0 Answers0