0

I am trying to call mmap with a direct syscall.

#include <sys/mman.h>

int main() {
    __asm__("mov $0x0, %r9;"
            "mov $0xffffffffffffffff, %r8;"
            "mov $0x32, %rcx;"
            "mov $0x7, %rdx;"
            "mov $0x1000, %rsi;"
            "mov $0x303000, %rdi;"
            "mov $0x9, %rax;"
            "syscall;");
    return 0;
}

I compiled the program statically:

$ gcc -static -o foo foo.c

But the syscall fails, as shown by strace:

$ strace ./foo
mmap(0x303000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE, -1, 0) = -1 EBADF (Bad file descriptor)

We can see that mmap flags are wrongly set. 0x32 should be MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS. The thing is, if I put another mmap call with mmap from the libc:

int main() {
    mmap(0x202000, 4096, 0x7, 0x32, -1, 0);
    __asm__("mov $0x0, %r9;"
            "mov $0xffffffffffffffff, %r8;"
            "mov $0x32, %rcx;"
            "mov $0x7, %rdx;"
            "mov $0x1000, %rsi;"
            "mov $0x303000, %rdi;"
            "mov $0x9, %rax;"
            "syscall;");
    return 0;
}

Then both mmap work:

$ strace ./foo
mmap(0x202000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x202000
mmap(0x303000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x303000

So it seems that using libc, mmap flags are "resolved" or something. But I can't really understand what is happening.

Why does the mmap syscall example only work if I put a libc mmap call before ?

Grominet
  • 23
  • 2

1 Answers1

1

Kernel syscall interface on AMD64 uses r10 register as a fourth argument, not rcx.

mov $0x32, %r10

See linked question for more details.

zch
  • 14,931
  • 2
  • 41
  • 49
  • That's it, thanks! And now I understand why it was working with the libc call before: r10 was set and didn't change between the two calls. – Grominet Feb 16 '19 at 17:23