2

I'm attempting to create a square root function with my double sqrt(double num) where num is the number that will be square rooted. Let's say I input a 25.25 into my function. The expected output will be 5.02493781056 according to my calculator. However, somehow is getting a segmentation fault. The function is purely for learning purposes. I was browsing on the internet saying that I need to return the full value of the memory address that I'm returning. Here is what I have. The code snippet is running with nasm -f macho64 with a main.c file.

sqrt.s 

section .text
global _sqrt

; double sqrt(double num);
_sqrt:
    fld     qword [rdi]     ; read the number given
    fsqrt                   ; float square root instruction
    fst     qword [rax]     ; float store value to rax
    ret                     ; and return it's value

main.c

int main(void)
{
  printf("my square function return: %f", sqrt(25));
  return (0);
}
Zeid Tisnes
  • 396
  • 5
  • 22
  • 1
    x86-64 System V passes the first FP arg in xmm0 so `sqrtsd xmm0, xmm0` / `ret`. No calling convention passes `double` by reference; 32-bit would pass it on the stack directly. I'm surprised `rax` happened to hold a valid pointer so your `fst` didn't fault. – Peter Cordes Aug 13 '18 at 02:58
  • In 64-bit code floats and doubles are passed in the vector registers (first float or doubleis passed in XMM0. Doubles and floats are returned through XMM0 (not RAX) – Michael Petch Aug 13 '18 at 02:59
  • Oh I misread, you *are* getting a segfault. Single-step through the compiler-generated caller in our debugger if you want to see where it puts your args. – Peter Cordes Aug 13 '18 at 03:24
  • Hi, it seems to be stopping at the first instruction. However, I'm replacing the `fld qword [rdi]` to `sqrtsd xmm0, xmm0` in the first instruction but still segfaulting. UPDATE: having `sqrtsd xmm0, xmm0 / ret` is giving me a value of 0.00000 @PeterCordes – Zeid Tisnes Aug 14 '18 at 00:54
  • @ZeidTisnes: Is the next instruction `ret`? If not, you're doing it wrong, and of course it still segfaults when you try to use `[rax]` as a pointer, because the caller doesn't pass a pointer in RAX. **Use your debugger to see which instruction faults**. Look at the first code block in John's answer on [Assembly 64bit: How to return a double value?](https://stackoverflow.com/a/37789295), and [Calling Convention of Floats in Nasm](https://stackoverflow.com/q/40916438). – Peter Cordes Aug 14 '18 at 01:01
  • If you're getting `0.0`, what value is in `xmm0` when your function is called? Or look at the compiler-generated caller and see where it's putting the `25.0`. Did you try naming your function something other than `sqrt`, which is already the name of a standard library function? **Did you include a prototype for `double sqrt(double)` in your `main.c`**? If not, that would explain it. Unprototyped functions default to returning `int`, like the compiler warnings tell you. – Peter Cordes Aug 14 '18 at 01:06

0 Answers0