7

The following on my Mac succeeds:

int main() {
    int* addr = (int*) mmap(0, 100, 1 | 2, 2 | 4096, -1, 0);

    *addr = 25;

    return 0;
}

However the below code is identical but fails when I try to write to *addr with segmentation fault:

int main() {
    int* addr = (int*) syscall(SYS_mmap, 0, 100, 1 | 2, 2 | 4096, -1, 0);

    *addr = 25;

    return 0;
}

I.e. syscall successfully returns me a memory address, but when I try writing to it it fails.

I compile it like this:

g++ ./c++/mmap.cc -o ./mmap && ./mmap

If I run both versions with dtruss:

g++ ./c++/mmap.cc -o ./mmap && sudo dtruss ./mmap

then both version succeed and I see identical mmap call for both:

mmap(0x0, 0x64, 0x3, 0x1002, 0xFFFFFFFF, 0x0)            = 0xXXXXXXX 0

Why does the syscall version give me segmentation fault, what am I missing?

P.S. If I do something similar on Linux it works fine.

So, as I understand the mmap function on Mac does not execute syscall(SYS_mmap, .... What does it do then? Can anyone please give me some links where I can see implementation.

EDIT:

It looks like syscall on Mac returns only first 4 bytes. Is there a 64-bit syscall version?

DISASSEMBLED:

mmap version:

_main:
0000000100000cf0        pushq   %rbp
0000000100000cf1        movq    %rsp, %rbp
0000000100000cf4        subq    $0x30, %rsp
0000000100000cf8        xorl    %eax, %eax
0000000100000cfa        movl    %eax, %ecx
0000000100000cfc        movl    $0x64, %eax
0000000100000d01        movl    %eax, %esi
0000000100000d03        movl    $0x3, %edx
0000000100000d08        movl    $0x1002, %eax
0000000100000d0d        movl    $0xffffffff, %r8d
0000000100000d13        movl    $0x0, -0x14(%rbp)
0000000100000d1a        movq    %rcx, %rdi
0000000100000d1d        movq    %rcx, -0x28(%rbp)
0000000100000d21        movl    %eax, %ecx
0000000100000d23        movq    -0x28(%rbp), %r9
0000000100000d27        callq   0x100000ed6 ## symbol stub for: _mmap
0000000100000d2c        movq    0x2cd(%rip), %rdi ## literal pool symbol address: __ZNSt3__14coutE
0000000100000d33        movq    %rax, -0x20(%rbp)
0000000100000d37        movq    -0x20(%rbp), %rax
0000000100000d3b        movq    %rax, %rsi

syscall version:

_main:
0000000100000cf0        pushq   %rbp
0000000100000cf1        movq    %rsp, %rbp
0000000100000cf4        subq    $0x30, %rsp
0000000100000cf8        movl    $0xc5, %edi
0000000100000cfd        xorl    %esi, %esi
0000000100000cff        movl    $0x64, %edx
0000000100000d04        movl    $0x3, %ecx
0000000100000d09        movl    $0x1002, %r8d
0000000100000d0f        movl    $0xffffffff, %r9d
0000000100000d15        movl    $0x0, -0x14(%rbp)
0000000100000d1c        movl    $0x0, (%rsp)
0000000100000d23        movb    $0x0, %al
0000000100000d25        callq   0x100000ed6 ## symbol stub for: _syscall
0000000100000d2a        movq    0x2cf(%rip), %rdi ## literal pool symbol address: __ZNSt3__14coutE
0000000100000d31        movslq  %eax, %r10
0000000100000d34        movq    %r10, -0x20(%rbp)
0000000100000d38        movq    -0x20(%rbp), %r10
0000000100000d3c        movq    %r10, %rsi
Vad
  • 4,052
  • 3
  • 29
  • 34
  • Why do people want to close this? Possibly because it belongs more on a system programming forum. But I don't (fully) agree. – meaning-matters Dec 24 '17 at 11:52
  • @meaning-matters exactly, why? This is a good question. – Vad Dec 24 '17 at 11:53
  • @meaning-matters Asking for recommendations/links. See the [FAQ]. Also, why invoking a `SYS_mmap` which is linux specific on a mac? – t0mm13b Dec 24 '17 at 11:54
  • May you post the disassembly of the second bit? – edmz Dec 24 '17 at 11:56
  • @t0mm13b: syscall is also perfectly valid on OS X. `man syscall`: "This function is useful for testing new system calls that do not have entries in the C library." – Jongware Dec 24 '17 at 11:56
  • 3
    @t0mm13b It's a pure programming question that ends with a very typical/correct SO question `Why does the syscall version give me segmentation fault, what am I missing?`. Only the second question is asking for links, which only proves that Vad is willing to sort it our himself; very good! – meaning-matters Dec 24 '17 at 11:57
  • `SYS_mmap` is not present in the OS X's file -> see [this](https://opensource.apple.com/source/xnu/xnu-1504.3.12/bsd/kern/syscalls.master) – t0mm13b Dec 24 '17 at 11:58
  • 1
    @t0mm13b SYS_mmap on Mac evaluates to 197, also in your link if you see 197 syscall, it is indeed mmap – Vad Dec 24 '17 at 12:00
  • @edmz posted disassembly – Vad Dec 24 '17 at 12:08
  • BSD Kernel code, have you compared to Linux, how it handles the mmap function? Comparing to Linux and saying it works ok under Linux is rather leading you to a false sense of security in thinking that it should work on Mac. Maybe there's some macro magic happing on the syscall which invokes the kernel's implementation? – t0mm13b Dec 24 '17 at 12:08
  • Compiling your code with `-m32` (and `clang`, though it should not make a difference) makes it work. – Jongware Dec 24 '17 at 12:09
  • 2
    Perhaps read `man syscall` on the mac. – n. m. could be an AI Dec 24 '17 at 12:09
  • 1
    @usr2564301 - The return type of `syscall` is `int`. – Oliver Charlesworth Dec 24 '17 at 12:34
  • @OliverCharlesworth: and *that* is a valid answer, at last! – Jongware Dec 24 '17 at 12:36
  • @OliverCharlesworth is there `syscall64` or something like that that returns `long`? I tried `__syscall`, returns `undeclared identifier`. – Vad Dec 24 '17 at 12:37
  • @usr2564301 - Alas, I don't know how one would work around this. i.e. what the 64-bit equivalent of `syscall` is. – Oliver Charlesworth Dec 24 '17 at 12:37
  • See https://stackoverflow.com/questions/38640828/c-syscall-64-bit-pointer – Oliver Charlesworth Dec 24 '17 at 12:40
  • 1
    On Linux `syscall` is 64-bit. How do FreeBSD people execute 64-bit syscalls? – Vad Dec 24 '17 at 12:40
  • Wait, that question is *also* yours, just from a year ago. Mysterious. Do you attempt to solve this problem once per year? – Oliver Charlesworth Dec 24 '17 at 12:43
  • @OliverCharlesworth that question was on Linux. Now its Mac. :D – Vad Dec 24 '17 at 12:46
  • [This related overview](https://stackoverflow.com/questions/46087730/what-happens-if-you-use-the-32-bit-int-0x80-linux-abi-in-64-bit-code) suggests `unistd_64.h` should be used. I don't see it but I have a pretty old OS – perhaps it's included on newer system? – Jongware Dec 24 '17 at 12:56
  • @usr2564301 no, I don't have it either – Vad Dec 24 '17 at 13:01

1 Answers1

5

Apparently Mac does not have a 64-bit syscall function, here a is simple implementation:

#include <sys/types.h>

#define CARRY_FLAG_BIT 1

inline int64_t syscall6(int64_t num, int64_t arg1, int64_t arg2, int64_t arg3, int64_t arg4, int64_t arg5, int64_t arg6) {
    int64_t result;
    int64_t flags;

    __asm__ __volatile__ (
        "movq %6, %%r10;\n"
        "movq %7, %%r8;\n"
        "movq %8, %%r9;\n"
        "syscall;\n"
        "movq %%r11, %1;\n"
        : "=a" (result), "=r" (flags)
        : "a" (num), "D" (arg1), "S" (arg2), "d" (arg3), "r" (arg4), "r" (arg5), "r" (arg6)
        : "%r10", "%r8", "%r9", "%rcx", "%r11"
    );

    return (flags & CARRY_FLAG_BIT) ? -result : result;
}

And you use it on mac by shifting system call numbers by 0x2000000:

int* addr = (int*) syscall6(0x2000000 + SYS_mmap, 0, 100, 1 | 2, 2 | 4096, -1, 0);

You can find more here.

Vad
  • 4,052
  • 3
  • 29
  • 34
  • 2
    Nice work! A minor note on terminology: it's not the *syscall* number that gets shifted, but its (required) *class*. See http://dustin.schultz.io/mac-os-x-64-bit-assembly-system-calls.html. As it is required on OS X, you could fit that into your calling routine. – Jongware Dec 24 '17 at 22:38
  • 1
    I wouldn't put too much trust into `"a"(num)` being placed to `*ax` as this is only true on x86 IIRC. You'd better have an explicit `movq %2, %%rax`, IMHO. – edmz Dec 26 '17 at 19:35
  • @edmz: specific-register constraints like `"a"` work reliably on x86-64 exactly the same way they do on i386. https://gcc.gnu.org/onlinedocs/gcc/Machine-Constraints.html. (Except that `"=A"` unfortunately will pick RDX *or* RAX instead of EDX:EAX for a 64-bit operand. It means RDX:RAX for a 128-bit operand, though). There aren't specific-register constraints for r8..r15, but you can use `register int foo asm("r10");` [Constraining r10 register in gcc inline x86\_64 assembly](//stackoverflow.com/q/15997759) – Peter Cordes Jul 09 '19 at 04:40
  • This asm statement still only needs just the `"syscall"` instruction + appropriate asm constraints. But **it's missing a `"memory"` clobber**: a store into a buffer before a `write()` system call made this way could optimize away as a dead store or reorder with this, because you haven't told the compiler that anything reads that store. – Peter Cordes Jul 09 '19 at 04:42
  • You could also use GCC6 flag-output syntax to let gcc make code that branches on CF directly (instead of the copy of RFLAGS in R11). But clang still doesn't support that syntax, unfortunately, so probably not a good idea. Using R11 is a good idea as long as OS X *always* updates R11 with the current flags during the return to user-space, even if it chooses to use `iret` instead of `sysret`. (Like Linux does if ptrace has been used, to work around the Intel CPU bug that would let a non-canonical return-RIP set by a debugger cause a #GP exception in ring 0 with RSP pointing to the user stack!) – Peter Cordes Jul 09 '19 at 04:47