0

I am diving into x86_64 assembly on macOS 10.12.

I am trying to call libc exit() function:

.section __TEXT,__text
.globl _main
_main:
    pushq %rbp
    movq %rsp, %rbp
    movl $5, %edi
    callq _exit

And compile it with:

as exit2.s -o exit2.o
ld exit2.o -e _main -lc -o exit

The result is:

Segmentation fault: 11

Why is it so?

Edit: the issue is in linking libc and calling conventions.

CorellianAle
  • 645
  • 8
  • 16
  • Try using your C compiler to link so you get all the necessary stuff linked in correctly. – Jester Jan 15 '18 at 21:10
  • @Jester I am using [this guide](http://www.idryman.org/blog/2014/12/02/writing-64-bit-assembly-on-mac-os-x/) as a reference. – CorellianAle Jan 15 '18 at 21:11
  • From a glance, that guide has horrible things, I would not be surprised if this one didn't work either. Have you tried `gcc -o exit exit2.s` (or equivalent?) – Jester Jan 15 '18 at 21:13
  • Possible duplicate of [Call C/C++ function from assembly (OSX Mavericks x64)](https://stackoverflow.com/questions/23136530/call-c-c-function-from-assembly-osx-mavericks-x64) – Macmade Jan 15 '18 at 21:15
  • @Jester Yes, it works with `gcc`. Do I understand it right: `gcc` performs some under-the-hood linking magic? My goal was to understand what happens there. – CorellianAle Jan 15 '18 at 21:16
  • Most probably a stack-alignment issue. Read the above question and answer. – Macmade Jan 15 '18 at 21:16
  • Also: https://stackoverflow.com/questions/12678230/how-to-print-argv0-in-nasm – Macmade Jan 15 '18 at 21:17
  • 1
    Yeah it could also be a stack alignment (although I would be surprised if `exit` needed anything aligned). You could try removing the `push %rbp` and see if it works that way. Also you can do `gcc -v -o exit exit2.s` to get it to print what commands it is running for you. Obviously best would be to run it in a debugger so you can actually examine the fault. – Jester Jan 15 '18 at 21:17
  • @Jester The macOS ABI requires stack alignment when calling a C function. Period. It does not depend on which function you call. – Macmade Jan 15 '18 at 21:21
  • 2
    You don't initialize the libc, no wonder your program crashes. Never ever use the libc without also linking through the C compiler so the libc initialization stub is linked in. – fuz Jan 15 '18 at 21:22
  • @Macmade It does depend whether you get a crash or not. Because it's not actively checking for alignment so you can get lucky if the function does not use aligned instructions. – Jester Jan 15 '18 at 21:23
  • @Jester If this guide is bad, do you know any decent getting-started tutorial? – CorellianAle Jan 15 '18 at 21:25
  • @CorellianAle: Don't call your ELF entry point `_main` or `main` in a static executable. CRT startup code hasn't run yet. The usual convention is to call it `_start`. (Note that OS X puts the CRT start code in the dynamic linker, so [the "entry point" in a dynamically-linked executable *is* the C `main` function, unlike in Linux](https://stackoverflow.com/questions/47801580/can-i-do-ret-instruction-from-code-at-start-in-macos-linux). **So libc would be initialized if you linked with `gcc` instead of `ld` or `gcc -static -nostartfiles`**.) – Peter Cordes Jan 16 '18 at 09:47
  • I'm curious. When you link with LD do you get any warnings? – Michael Petch Jan 16 '18 at 10:36

1 Answers1

1

@fuz is almost certainly correct: you crash because you didn't initialize libc. There's probably a NULL pointer somewhere in the data structures that exit(3) checks before actually exiting. e.g. it flushes stdout if needed, and it runs any functions registered with atexit(3).

If you don't want it to do all that work, then either make the sys_exit system call directly with a syscall instruction, or call the thin _exit(2) libc wrapper function for it. (The basics of the situation will be the same as on Linux, because exit(3) vs. _exit(2) are standardized by POSIX: see Syscall implementation of exit().

I think the tutorial you're following mostly looks good, but perhaps some older version of OS X allowed libc functions (including printf?!?) to be used without calling any libc init functions. Or else they didn't test their code after an edit to the build commands. (Assuming they tested at all, maybe it was with dynamic linking, which would work.)

OS X prefixes symbol names in assembly with an _, so use call __exit (two underscores) to call _exit(). (e.g. call _printf calls the C printf function).

_exit(2) probably won't crash if you call it without initializing libc, but it's still a bad idea to call any libc functions without having called libc init functions first. Better to make the system call directly (see later in the tutorial), or even better, build it with gcc hello_asm.S -o hello_asm to make sure libc is initialized. Then you can follow the rest of the tutorial, including the printf.

Don't call your Mach-O entry point _main or main in a static executable. CRT startup code hasn't run yet. The usual convention is to call it _start for the process entry point.

(Note that OS X puts the CRT start code in the dynamic linker, so the "entry point" in a dynamically-linked executable is the C main function, unlike in Linux where dynamic executables can avoid the CRT startup code.

libc would be initialized for you if you linked with gcc exit2.o -o exit instead of ld, which you're using to do the equivalent of gcc -static -nostartfiles.)

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847