1

I really need some help here. I've been searching online for about 2 days now and can't seem to find an answer to the problems that I got.

  1. I downloaded nasm and installed it it seems to work but I can't seem to find any example code that works on windows 64bit (it would be awesome if you could make a simple hello world example for me or something). (And how to compile it)

  2. How would I turn the .o file into a .exe?

  3. How would I accesses the screen or gpu memory (I think) to print stuff on the screen (like some sort of console application)?

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
Knowhow
  • 31
  • 1
  • 4
  • *"How would I accesses the screen or gpu memory"*: not directly as in DOS age, you have to call Windows API to get access to these, which usually tends to be quite cumbersome in Assembly. Unless you want to do an 4k demo, I would consider using C++ initialization to call all the win API to prepare screen/gpu memory/etc... and give your assembly simple pointer + data to toy with it in assembly. If you want to avoid clib, then you will have to do lot of the API calling yourself, may involve also loading additional DLLs into memory and dynamically linking to their functions: pages of source... – Ped7g Dec 16 '16 at 09:19

2 Answers2

10

There a a few things to consider when programming in assembly using NASM under Windows:

  1. NASM is just an assembler
    A linker is needed to create an executable.

  2. Windows API are implemented as a user space library.
    The library is not entirely in user space, some functions cross the boundary, but the interface to the application is a set of DLL, most notably: kernel32.dll, user32.dll and so on.

  3. Windows' executables can have different subsystems.
    Among the others the IMAGE_SUBSYSTEM_WINDOWS_GUI and IMAGE_SUBSYSTEM_WINDOWS_CUI are the most used, the former being used by applications that create windows and the latter by applications that use a console1.


The linker takes to COFF object (.obj) file generated by the assembler, takes a set of library definitions (.lib) and perform these important steps:

  1. It resolves the call to external functions found in the object file.
    A function marked as external is meant to be found in one of the library definition, the linker then fixes the call instruction to point to a location where the address of the function will be found at run-time.

  2. It add the dependency libraries to the final executable.
    This is done with the help of the PE file format, basically if F if found on the library L then L is added to the dependencies list along with a location where to write the address of F.
    The loader will do the rest.

The linking is actually more complex than that.
It is also responsible for setting the various PE flags and settings, including the subsytem and the entrypoint.


To write an executable using NASM with need:

  1. A linker.
  2. The set of library definitions.

These can be found in the Windows SDK2.

The linker we will use in this example is the Microsoft LINK linker.

As @CodyGray pointed out in its comment (quoting because I'm lazy):

The Windows 7 version of the SDK that you linked does include all of the build tools. The Windows 8 SDK changed and no longer ships with the command-line build tools, forcing you to download a Visual Studio package


Once we have the tools, we can start programming.
We will use WriteFile to write a string to the standard output (like we would in Linux assembly).
The handler to the standard output must be retrieved with GetStdHandle though.

In Windows you can exit a process with a ret even with no CRT (C run-time) linked (i.e. in a bare process).

I won't explain the Window 64 ABI nor the trickery of programming in 64-bit.

BITS 64
DEFAULT REL                  ;RIP relative addressing by default

GLOBAL main                  ;Main must be visible to the linker

;Function the linker will look for in other modules (libs or objs)
EXTERN WriteFile
EXTERN GetStdHandle

STD_OUTPUT_HANDLE   EQU -11

SECTION .data

 strHelloWorld  db "Hello world!", 13, 10, 0
 lenHelloWorld  dd $-strHelloWorld

 hOut           dd 0    ;Will store STDOUT handler    

SECTION .text

main:
 sub rsp, 30h               ;20h (Home space) + 10h (params)


 ;Get STDOUT handler (Handler are 32-bit values)

 mov ecx, STD_OUTPUT_HANDLE
 call GetStdHandle

 mov DWORD [hOut], eax                 ;Useless as now, for future reuse

 ;Write strHelloWorld to STDOUT

 mov ecx, eax
 lea rdx, [strHelloWorld]
 mov r8d, DWORD [lenHelloWorld]
 xor r9, r9
 mov QWORD [rsp+20h], r9
 call WriteFile

 add rsp, 30h
 ret  

To create an executable from this source, we first assemble it into a 64-bit object file:

nasm -fwin64 hello.asm -o hello.obj

then we link it against the kernel32 library (the only one we need):

link /MACHINE:X64 /SUBSYSTEM:CONSOLE /OUT:hello.exe /NODEFAULTLIB /ENTRY:main kernel32.lib hello.obj

the CLI switches are pretty straightforward, NODEFAULTLIB avoid linking with MSVCVRXX.lib, the Microsoft CRT. Those are the minimal switches necessary to build an executable in this example.

I assume you are smart enough to fix the path issues when issuing the commands above, particularly you can tell link where to find the lib files.
For this example I just copied them from the SDK installation folder.


As proposed in other answers, you can use other assemblers that may came with a linker.
Another possible developing environment is to use gcc (either inside cygwin or as mingw) to perform the linker step.


1 In this case Windows automatically create a console for the application and set the standard handlers appropriately. GUI application can recreate the console too (or more than one), so IMAGE_SUBSYSTEM_WINDOWS_GUI must be intended as "No console by default".

2 The version linked is the first Google result, invest more time in finding a suitable version. Also, I'm not sure if there is a linker in the Windows SDK or you need to download a Visual Studio instead.

Margaret Bloom
  • 41,768
  • 5
  • 78
  • 124
  • 1
    The Windows 7 version of the SDK that you linked does include all of the build tools. The Windows 8 SDK changed and no longer ships with the command-line build tools, forcing you to download a Visual Studio package. :-( – Cody Gray - on strike Dec 16 '16 at 15:58
  • Thank you @CodyGray that's very useful info! I'm adding them to the answer. – Margaret Bloom Dec 16 '16 at 17:03
1

Download Flat Assembler.(Download->flat assembler for windows)

http://flatassembler.net/

Extract zip.

Goto 'examples/win64/pe64demo/'

open 'pe64demo.exe'

(source code is in 'pe64demo.asm')

Nadeen Udantha
  • 60
  • 1
  • 3
  • 9