Incorrect NASM indirect addressing assembly on macOS

Question

Assembling the following code on macOS:

global start

default rel

section .text

start:
    lea rdx, [buffer + 0]
    lea rdx, [buffer + 1]
    lea rdx, [buffer + 2]
    lea rdx, [buffer + 3]
    lea rdx, [buffer + 4]
    lea rdx, [buffer + 5]
    lea rdx, [buffer + 6]
    lea rdx, [buffer + 7]
    lea rdx, [buffer + 8]


section .data

buffer: db 0,0,0

using the command nasm -fmacho64 -w+all test.asm -o test.o, yields: (with gobjdump -d test.o)

0000000000000000 <start>:
   0:   48 8d 15 38 00 00 00    lea    0x38(%rip),%rdx        # 3f <buffer>
   7:   48 8d 15 32 00 00 00    lea    0x32(%rip),%rdx        # 40 <buffer+0x1>
   e:   48 8d 15 2c 00 00 00    lea    0x2c(%rip),%rdx        # 41 <buffer+0x2>
  15:   48 8d 15 26 00 00 00    lea    0x26(%rip),%rdx        # 42 <buffer+0x3>
  1c:   48 8d 15 20 00 00 00    lea    0x20(%rip),%rdx        # 43 <buffer+0x4>
  23:   48 8d 15 1a 00 00 00    lea    0x1a(%rip),%rdx        # 44 <buffer+0x5>
  2a:   48 8d 15 14 00 00 00    lea    0x14(%rip),%rdx        # 45 <buffer+0x6>
  31:   48 8d 15 0e 00 00 00    lea    0xe(%rip),%rdx        # 46 <buffer+0x7>
  38:   48 8d 15 08 00 00 00    lea    0x8(%rip),%rdx        # 47 <buffer+0x8>

This looks correct. When then linking that into an executable using ld test.o -o test, we get: (with gobjdump -d test)

0000000000001fc1 <start>:
    1fc1:   48 8d 15 38 00 00 00    lea    0x38(%rip),%rdx        # 2000 <buffer>
    1fc8:   48 8d 15 32 00 00 00    lea    0x32(%rip),%rdx        # 2001 <buffer+0x1>
    1fcf:   48 8d 15 2c 00 00 00    lea    0x2c(%rip),%rdx        # 2002 <buffer+0x2>
    1fd6:   48 8d 15 23 00 00 00    lea    0x23(%rip),%rdx        # 2000 <buffer>
    1fdd:   48 8d 15 1d 00 00 00    lea    0x1d(%rip),%rdx        # 2001 <buffer+0x1>
    1fe4:   48 8d 15 17 00 00 00    lea    0x17(%rip),%rdx        # 2002 <buffer+0x2>
    1feb:   48 8d 15 11 00 00 00    lea    0x11(%rip),%rdx        # 2003 <buffer+0x3>
    1ff2:   48 8d 15 0b 00 00 00    lea    0xb(%rip),%rdx        # 2004 <buffer+0x4>
    1ff9:   48 8d 15 05 00 00 00    lea    0x5(%rip),%rdx        # 2005 <buffer+0x5>

And suddenly, unexpectedly, buffer + n gets changed into buffer + (n - 3) if n >= 3. The nasm version I'm using is 2.13.01 and the ld is macOS Sierra's system linker, with ld -v giving:

@(#)PROGRAM:ld  PROJECT:ld64-274.2
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
LTO support using: LLVM version 8.0.0, (clang-800.0.42.1)
TAPI support using: Apple TAPI version 1.30

Why is this happening? Note that the same code seems to assemble and link fine on linux, as well as using yasm. In other places I read that nasm has some weird problems with the Mach-O 64 format, so this is probably a nasm bug in that area.

Wild guess at the moment, what is the result when you extend the declaration of buffer to 8 bytes? — Frank C., Jul 14 '17 at 10:02
@FrankC. Changing the last line to `buffer: db 0,0,0,0, 0,0,0,0` restarts counting from `buffer+0x8` (as extrapolated from the data above); changing it to `buffer: dq 0` gives exactly the same result. In general, regardless of the size of `buffer`, it seems to start re-counting from the end of the buffer, but doesn't restart anymore after that. — tomsmeding, Jul 14 '17 at 18:23
Note that the same code works absolutely fine using [yasm](http://yasm.tortall.net/), which leads me to think this is indeed a nasm bug. — tomsmeding, Jul 14 '17 at 18:23
There's [a NASM bug with 2.13.02+ on OS X](https://stackoverflow.com/questions/49224997/successive-sys-write-syscalls-not-working-as-expected-nasm-bug-on-os-x), but MichaelPetch says 2.13.01 should be fine. Also, the known bug didn't affect RIP-relative this time, that was [the NASM 2.11.08 + macho64 bug](https://stackoverflow.com/questions/30814930/nasm-issue-on-osx-64-bit). Have you reported this? — Peter Cordes, Apr 20 '18 at 08:32
@PeterCordes No, I didn't report it yet. But I just tested again using versions 2.13.0{1,2,3} of nasm downloaded from [here](https://www.nasm.us/pub/nasm/releasebuilds/?C=M;O=D); 2.13.01 gives the same results as in the question (luckily it's reproducible), .02 gives really strange output for 'test.o' but a correct 'test', and .03 gives the same outputs as .02. So apparently, this has been fixed already. — tomsmeding, Apr 20 '18 at 13:21
Hmm, well maybe it's related to the 2.13.01 bug, or your 2.13.02 wasn't *really* a 2.13.02 release version and still had the .01 bug? Thanks for checking. — Peter Cordes, Apr 20 '18 at 13:31
The "strange" output for the `.o` is just because you haven't linked yet, and the disassembler is only using the placeholder values in 32-bit field. If you used `objdump -d -r` to see relocations, you should see correct stuff from the `.o`. (I use `alias disas='objdump -drwC -Mintel'`) — Peter Cordes, Apr 20 '18 at 13:34
@PeterCordes For the .02 version, I downloaded [this one](https://www.nasm.us/pub/nasm/releasebuilds/2.13.02/macosx/), which as far as I can tell is a normal 2.13.02. And thanks for the hints about objdump. :) — tomsmeding, Apr 20 '18 at 18:17

tomsmeding · Answer 1 · 2018-04-20T22:54:48.103

This seems to be fixed in nasm version 2.13.03, as tested today. (Also in 2.13.02, but that version has other problems with the macho64 target anyway, re @MichaelPetch; .02's output is equal in this case to .03's.) The output is as follows for the test.asm and commands given in the question.

nasm -fmacho64 -w+all test.asm -o test.o; gobjdump -d test.o:

test.o:     file format mach-o-x86-64


Disassembly of section .text:

0000000000000000 <start>:
   0:   48 8d 15 00 00 00 00    lea    0x0(%rip),%rdx        # 7 <start+0x7>
   7:   48 8d 15 01 00 00 00    lea    0x1(%rip),%rdx        # f <start+0xf>
   e:   48 8d 15 02 00 00 00    lea    0x2(%rip),%rdx        # 17 <start+0x17>
  15:   48 8d 15 03 00 00 00    lea    0x3(%rip),%rdx        # 1f <start+0x1f>
  1c:   48 8d 15 04 00 00 00    lea    0x4(%rip),%rdx        # 27 <start+0x27>
  23:   48 8d 15 05 00 00 00    lea    0x5(%rip),%rdx        # 2f <start+0x2f>
  2a:   48 8d 15 06 00 00 00    lea    0x6(%rip),%rdx        # 37 <start+0x37>
  31:   48 8d 15 07 00 00 00    lea    0x7(%rip),%rdx        # 3f <buffer>
  38:   48 8d 15 08 00 00 00    lea    0x8(%rip),%rdx        # 47 <buffer+0x8>

ld test.o -o test; gobjdump -d test:

test:     file format mach-o-x86-64


Disassembly of section .text:

0000000000001fc1 <start>:
    1fc1:       48 8d 15 38 00 00 00    lea    0x38(%rip),%rdx        # 2000 <buffer>
    1fc8:       48 8d 15 32 00 00 00    lea    0x32(%rip),%rdx        # 2001 <buffer+0x1>
    1fcf:       48 8d 15 2c 00 00 00    lea    0x2c(%rip),%rdx        # 2002 <buffer+0x2>
    1fd6:       48 8d 15 26 00 00 00    lea    0x26(%rip),%rdx        # 2003 <buffer+0x3>
    1fdd:       48 8d 15 20 00 00 00    lea    0x20(%rip),%rdx        # 2004 <buffer+0x4>
    1fe4:       48 8d 15 1a 00 00 00    lea    0x1a(%rip),%rdx        # 2005 <buffer+0x5>
    1feb:       48 8d 15 14 00 00 00    lea    0x14(%rip),%rdx        # 2006 <buffer+0x6>
    1ff2:       48 8d 15 0e 00 00 00    lea    0xe(%rip),%rdx        # 2007 <buffer+0x7>
    1ff9:       48 8d 15 08 00 00 00    lea    0x8(%rip),%rdx        # 2008 <buffer+0x8>

Worth noting is that my ld has updated and ld -v now gives:

@(#)PROGRAM:ld  PROJECT:ld64-305
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em (tvOS)
LTO support using: LLVM version 9.0.0, (clang-900.0.39.2) (static support for 21, runtime is 21)
TAPI support using: Apple TAPI version 900.0.15 (tapi-900.0.15)

2.13.02 versions of NASM aren't even recommended because they have their own set of problem with macho64 as well: https://bugzilla.nasm.us/show_bug.cgi?id=3392468 — Michael Petch, Apr 20 '18 at 20:44
@MichaelPatch Thanks, noted in the answer. I seem to remember problems as well, now I think of it. Haven't worked with nasm in a while anymore. — tomsmeding, Apr 20 '18 at 22:55

Incorrect NASM indirect addressing assembly on macOS

1 Answers1

Linked