Getting return value of syscall in assembly

Question

I'm writing a simple assembly program on Darwin x86_64 (MacOS 10.14.6) that creates a file (test.txt) and writes "E" to it. However, for some reason, the "E" is not written to the file. What am I doing wrong?

Here's my program:

global start
section .text
    start:
        ;Create the file
        mov rax, 0x2000005
        mov rdi, fname
        mov rsi, 0x200
        mov rdx, 0o644
        syscall

        ;Write to file
        mov rdi, rax     ;As far as I know, this uses the fd returned by creating a file (previous syscall)
        mov rsi, msg
        mov rdx, 1
        mov rax, 0x2000004
        syscall

        ;Exit
        mov rax, 0x2000001
        mov rdi, 0
        syscall
section .data
    msg: db "E"     ;Message
    fname: db "test.txt"     ;File name

I have also tried this:

global start
section .text
    start:
        mov rax, 0x2000005
        mov rdi, fname
        mov rsi, 0x200
        mov rdx, 0o644
        syscall
        mov rdi, rax
        mov rsi, msg
        mov rdx, 2
        mov rax, 0x2000004
        syscall
        mov rax, 0x2000001
        mov rdi, 0
        syscall
section .data
    msg: db "E", -1
    fname: db "test.txt", 0

Yet neither work.

MacOS syscall return values are in RAX, with CF indicating whether it's an error code or normal. You can use `dtrace` on your program to trace syscalls and decode that for you. — Peter Cordes, Apr 04 '20 at 19:54
I notice there's no `, 0` terminator on your fname string. If there are other bytes after it after linking, that's a problem. — Peter Cordes, Apr 04 '20 at 19:55
@PeterCordes And if there are not, this could cause the `open` call to return `EFAULT`. — fuz, Apr 04 '20 at 20:00
The file is successfully created, just nothing is written to it. but I will try your advice. — a_random_programmer, Apr 04 '20 at 20:02
Then use `dtrace` or a debugger to look at the return value from the `write` system call, obviously. (And verify the args). That's always the first step in figuring out why a system call didn't do what you wanted. — Peter Cordes, Apr 04 '20 at 20:04
@a_random_programmer What does dtrace say? Try to execute your program under `dtrace` and see if that works. E.g. type `dtrace foo` where `foo` is the name of your program. — fuz, Apr 04 '20 at 20:20
@fuz I got this as the result: `dtrace: no probes specified`. I tried with and without sudo — a_random_programmer, Apr 04 '20 at 20:21
@fuz I get this error: `dtrace cannot control executables signed with restricted entitlements` — a_random_programmer, Apr 04 '20 at 20:30
@a_random_programmer Ah how much I hate modern macOS. Refer to [this answer](https://stackoverflow.com/questions/33476432/is-there-a-workaround-for-dtrace-cannot-control-executables-signed-with-restri) for how to make it work. — fuz, Apr 04 '20 at 21:13

a_random_programmer · Answer 1 · 2020-04-04T22:11:02.907

I found the answer:

I found that opening the file with create mode and write mode works.

global start
section .text
    start:
        ;Open file with create mode and write mode
        mov rax, 0x2000005
        mov rdi, fname
        mov rsi, 0x201
        mov rdx, 0o644
        syscall

        ;Now write
        mov rdi, rax
        mov rsi, msg
        mov rdx, 1
        mov rax, 0x2000004
        syscall

        ;Exit
        mov rax, 0x2000001
        mov rdi, 0
        syscall
section .data
    msg: db "E"
    fname: db "test.txt"

Also, here is an array of flags for opening files:

O_ACCMODE: 0x3
O_APPEND: 0x8
O_ASYNC: 0x40
O_CLOEXEC: 0x1000000
O_CREAT: 0x200
O_DIRECTORY: 0x100000
O_DSYNC: 0x400000
O_EXCL: 0x800
O_EXLOCK: 0x20
O_NDELAY: 0x4
O_NOCTTY: 0x20000
O_NOFOLLOW: 0x100
O_NONBLOCK: 0x4
O_RDONLY: 0x0
O_RDWR: 0x2
O_SHLOCK: 0x10
O_SYNC: 0x80
O_TRUNC: 0x400
O_WRONLY: 0x1

To combine, use the | operator (in C) or or (in assembly).

Thanks for writing this up! Yeah, that's an easy mistake to make. If you don't open the file for writing, you are not permitted to write. — fuz, Apr 05 '20 at 16:45

Getting return value of syscall in assembly

1 Answers1