How to use x64 assembly with C++ on windows?

Question

cpp.cpp:

#include <iostream>

extern "C" int returnnum();

int main() { std::cout << returnnum() << "\n"; }

asm.asm:

segment .text
global _returnnum
_returnnum:
    mov rax, 420
    ret

First, I compile the assembly file with nasm -f win64 asm.asm.

Then, I compile the C++ file with g++ cpp.cpp asm.obj.

But this gives me an error:

asm.obj: file not recognized: File format not recognized
collect2.exe: error: ld returned 1 exit status

I know that I can change rax to eax and compile with -f win32 and it'll work, but I want a 64-bit application. Is there any way I can do this?

If this is helpful:

g++ --version: g++ (MinGW.org GCC-6.3.0-1) 6.3.0
nasm --version: NASM version 2.15rc12 compiled on Jun 26 2020

I think it would be helpful to add the exact compiler version and distribution you used. — user17732522, Jun 02 '22 at 16:24
Are you sure your `g++` is a 64 bit build tool, rather than 32 bit? Wondering if it might be defaulting to compiling your `.cpp` in 32 bit mode and getting confused by the use of a 64 bit object file. — ShadowRanger, Jun 02 '22 at 16:24
compile the cpp.cpp first and then only link the object files — Paweł Łukasik, Jun 02 '22 at 16:24
@ShadowRanger How do I check that? Adding `-m64` throws ```cpp.cpp:1:0: sorry, unimplemented: 64-bit mode not compiled in #include ``` — avighnac, Jun 02 '22 at 16:26
@PawełŁukasik `ld` throws `asm.obj: file not recognized: File format not recognized` — avighnac, Jun 02 '22 at 16:28
I'd use https://jmeubank.github.io/tdm-gcc/ . Download the one on the left that is mingw-w64 based. It has a 64-bit tool chain. — Michael Petch, Jun 02 '22 at 16:30
@avighnac: Sounds like you got your answer; your `g++` doesn't implement 64 bit targets. Guessing you installed MinGW not MinGW-W64? — ShadowRanger, Jun 02 '22 at 16:30
You may want to use msys2 and install mingw64 with gcc-11.3: [https://www.msys2.org/#installation](https://www.msys2.org/#installation) also [https://stackoverflow.com/questions/30069830/how-to-install-mingw-w64-and-msys2](https://stackoverflow.com/questions/30069830/how-to-install-mingw-w64-and-msys2) — drescherjm, Jun 02 '22 at 16:38

score 4 · Accepted Answer · answered Jun 02 '22 at 16:34

Based on your information from the comments and your g++ --version output, it looks like you installed plain MinGW (which is purely 32 bit, and works just fine if you only need to build 32 bit executables since Windows supports running 32 bit executables on a 64 bit OS) rather than MinGW-W64, the fork with native support for 64 bit Windows.

You'll need to switch if you want to build true 64 bit executables.

How to use x64 assembly with C++ on windows?

1 Answers1