2

I was trying to learn NASM and started from a helloworld program. The tutorial itself is 32 bit NASM on Linux.

I copy and paste the created helloworld.asm (using int 0x80 32-bit system calls) and input the commands...

nasm -f elf helloworld.asm
ld -m elf_i386 helloworld.o -o helloworld

these two looks good but running ./helloworld produces

-bash: ./helloworld: cannot execute binary file: Exec format error

Then I searched this error and fond this SO posting. I entered the commands of that answer:

sudo apt-get install gcc-multilib g++-multilib

after that, I installed this:

nasm -f elf64 helloworld.asm -o helloworld.o
ld -o helloworld helloworld.o -m elf_x86_64

both came from that answer and no error occurred.
Then I executed ./helloworld and got the following error:

Segmentation fault (core dumped)

Okay, then I searched this new error as well. (Editor's note: What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code? explains the exact reason: WSL 1 doesn't support 32-bit int 0x80 system calls in 64-bit code, and it's generally not a good idea).

Most of the answer doesn't give an exact solution. The only one I got is this. It looks like I need to modify the NASM code, he says

64bit sys_exit = 60 32bit sys_exit = 1
64bit sys_write = 1 32bit sys_write = 4

and

32 bit sys_exit:
mov     ebx, ERR_CODE
mov     eax, sys_exit  ; 1
int     80h
64 bit sys_exit:
mov     rdi, ERR_CODE
mov     rax, sys_exit  ; 60
syscall

so I modified the original code and it became this:
The original code:

; Hello World Program - asmtutor.com
; Compile with: nasm -f elf helloworld.asm
; Link with (64 bit systems require elf_i386 option): ld -m elf_i386 helloworld.o -o helloworld
; Run with: ./helloworld
 
SECTION .data
msg     db      'Hello World!', 0Ah
 
SECTION .text
global  _start
 
_start:
 
    mov     edx, 13
    mov     ecx, msg
    mov     ebx, 1
    mov     eax, 4
    int     80h
 
    mov     ebx, 0      ; return 0 status on exit - 'No Errors'
    mov     eax, 1      ; invoke SYS_EXIT (kernel opcode 1)
    int     80h

And the modified code:

; Hello World Program - asmtutor.com
; Compile with: nasm -f elf helloworld.asm
; Link with (64 bit systems require elf_i386 option): ld -m elf_i386 helloworld.o -o helloworld
; Run with: ./helloworld
 
SECTION .data
msg     db      'Hello World!', 0Ah
 
SECTION .text
global  _start
 
_start:
 
    mov     edx, 13
    mov     ecx, msg
    mov     rdi, 1
    mov     rax, 1      ; SYS_write 64-bit ABI
    syscall
 
    mov     rdi, 0      ; return 0 status on exit - 'No Errors'
    mov     rax, 60     ; invoke SYS_exit (kernel opcode 1)
    syscall

After I modified the code, I input the command above again:

nasm -f elf64 helloworld.asm -o helloworld.o
ld -o helloworld helloworld.o -m elf_x86_64
./helloworld

Good. There is no error anymore. But there is not output either, which should be "helloworld".


Then I back to first tried the command given in tutorial, to build a 32-bit executable from the source that worked in 64-bit mode:

nasm -f elf helloworld.asm

and its result is:

helloworld.asm:16: error: instruction not supported in 32-bit mode
helloworld.asm:17: error: instruction not supported in 32-bit mode
helloworld.asm:20: error: instruction not supported in 32-bit mode
helloworld.asm:21: error: instruction not supported in 32-bit mode

Then I searched this error again, but I cannot find a solution.
My machine is a 64bit Linux Ubuntu as a Windows-Subsystem-For-Linux in 64bit Windows 10.

How to run this helloworld program correctly?
Does this mean that a 32bit Linux NASM tutorial/program just cannot run on a 64bit Linux?
Or a 64-bit Windows10?
Or in a Windows-Subsystem-For-Linux?

What is the problem, and why do occur so many errors?
How should I avoid similar potential errors in the future?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
stary
  • 21
  • 2
  • 1
    You used the wrong register for the write syscall. Change `mov ecx, msg` to `mov rsi, msg`. – Jester Sep 12 '19 at 22:48
  • 2
    You can't execute 32 bit code on WSL (Windows Subsystem for Linux). That's why you get an *exec format error.* You can execute 64 bit code, but only using 64 bit system calls (as you already noticed). Assemble and link with `nasm -f elf64 helloworld.asm; ld helloworld.o -o helloworld` and you should be good to go. – fuz Sep 13 '19 at 01:05
  • I have changed the code and it works properly. Is that means i need to modify every single 32 bit nasm in the tutorial? Or it is possible to convert/allow 32bit code compile&link&run? Also, if WSL cant execute 32 bit code, can native Linux do? – stary Sep 13 '19 at 07:00
  • 64 bit calling convention and syscall numbering is different from 32 bit. Also obviously pointer sizes are different. There isn't a simple way to convert 32 bit to 64 bit. Native linux can execute 32 bit unless specifically disabled. – Jester Sep 13 '19 at 11:30
  • 1
    https://stackoverflow.com/q/46087730/4271923 .. you should never call 32b services from 64b executable. Most of the common linux installations will actually allow that in some limited way (see the Q+A above for many details) (but not WSL), but if you have 32b tutorial, you should build 32b executables. Which is not possible with WSL either. I think all the current linux distributions still support 32b binaries, although some are trying hard to deprecate that. For tutorial-like stuff it's probably good enough to install any linux in virtual machine (or pick some older 32bit ISO to be sure). – Ped7g Sep 13 '19 at 11:43
  • *"What is the problem"* - and about this one... well, technically x86 and x86_64 target platforms are like two different computers (and you have in modern x86 machine also third one, the original 16bit mode x86 based on 80286, this one is often used in legacy BIOS and boot process). As assembly is different per CPU, the 32bit x86 code needs full rewrite to be x86_64 assembly code. Specifically on x86-family it may be initially not so obvious, because the instruction mnemonics are shared between and most of the time you are writing identical or similar assembly source, but it's different enough – Ped7g Sep 13 '19 at 12:51
  • If you don't know asm yet, don't try to port your tutorial from 32 to 64 bit!! Use an emulator or VM to run real Linux; almost all distros build their Linux kernel with CONFIG_IA32_EMULATION enabled, unlike the WSL "kernel". – Peter Cordes Sep 17 '19 at 04:30

1 Answers1

1

QUICK FIX
Do you want compile for 32 bits or 64 bits?
Your program will work and show output by changing only the register ecx by rsi

; Hello World Program - asmtutor.com
; Compile with: nasm -f elf helloworld.asm
; Link with (64 bit systems require elf_i386 option): ld -m elf_i386 helloworld.o -o helloworld
; Run with: ./helloworld

SECTION .data
msg     db      'Hello World!',0Ah

SECTION .text
global  _start

_start:

    mov     edx, 13
    mov     rsi, msg
    mov     rdi, 1
    mov     rax, 1
    syscall

    mov     rdi, 0      ; return 0 status on exit - 'No Errors'
    mov     rax, 60      ; invoke SYS_EXIT (kernel opcode 1)
    syscall

Compile your program for 64 bits using nasm and ld

nasm -f elf64 helloworld.asm -o helloworld.o  
ld helloworld.o -o helloworld.elf  
  • 1
    `mov esi, msg` would work, too, since you're making a non-PIE executable. Symbol addresses are guaranteed to be in the low 32 bits of virtual address space in the default code-model. Similarly, `mov edi, 1` / `mov eax,1` would save code size if NASM didn't already optimize those for you. If you *did* want to put a possibly-64-bit symbol address into a register, the good way is `lea rsi, [rel msg]`. [How to load address of function or label into register](https://stackoverflow.com/q/57212012) – Peter Cordes Jan 25 '21 at 04:48
  • 1
    Note that WSL 1 is like a Linux kernel without CONFIG_IA32_EMULATION, so no `int 0x80` system calls in 64-bit mode, and no 32-bit ELF executables. So the answer to your "Do you want compile for 32 bits or 64 bits?" question needs to be "64 bit". – Peter Cordes Jan 25 '21 at 05:16
  • 3
    But really the biggest piece of advice is: generally don't try to port a tutorial until after you've finished going through it. Find a tutorial for the system you want to use, because asm is *highly* non-portable, as this question amply demonstrates. (It did make a good attempt at doing the necessary research to find 64-bit system call info, but took a bunch of wild guesses at doing things like assembling that 64-bit code back into a 32-bit executable. Learning some more about x86-64 would have made that a lot less confusing.) – Peter Cordes Jan 25 '21 at 05:19
  • Pete your knowledge is always welcome, i'd like you correct me everytime that I fail but it makes me a little bit hard understand all your content fast because is deep =). Thank you again – another character escaped Jan 25 '21 at 05:40