0

Recently I've simply been trying to learn some Assembly and I figured Nasm would be a good choice since I am more used to its intel_syntax. Anyways, after deliberately trying to find some example programs for 64-bit Windows, I found this SO post and attempted to use the code provided by the 2nd answer. The only thing is that the code was for x86 windows which is not my target platform. So I attempted to write the program for 64-bit Windows (Note the word attempted). I am fairly new to Assembly so my code is most likely not correct and I would appreciate it if anyone would point out such errors.

The problem I have encountered, however, is with linking. For some reason, when I try to link the object file, the gcc linker cannot the resolve the external symbols (i.e. The winapi functions) and I cannot figure out why it is doing that.

Here is my code (I commented every line so that I could try to understand what the program is doing but please feel free to edit this post and change anything if it is wrong).

section .text               ;Section of the src file that contains the actual runtime code
global WinMain@16           ;Makes the address WinMain global. @16 signifies 16 bytes of parameters

extern  _GetStdHandle@4     ;Assures compiler of an external definition of the declared function 'GetStdHandle()'
extern  _WriteFile@36       ;Assures compiler of an external definition of the declared function 'WriteFile()'
extern  _ExitProcess@4      ;Assures compiler of an external definition of the declared function 'ExitProcess()'

WinMain@16:                 ;Label marking the entry-point of the program (Windows expects 'WinMain' unless explicitly specified)
    mov rbp, rsp            ;Move Stack Pointer value equal to Base Pointer
    sub rsp, 4              ;Move Stack Pointer back (i.e. subtract) 4 bytes to store DWORD '-11' (defined by 'STD_OUTPUT_HANDLE')

    push -11                ;Push '-11' into stack
    call _GetStdHandle@4    ;Calls GetStdHandle(-11) since -11 has been pushed. @4 signifies 4 bytes of parameters
    mov rbx, rax            ;rax stores return value -> move that value to rbx

    push 0                  ;Push 0 (NULL) onto the stack (This is the last argument in the function)
    lea rax, [rbp - 4]      ;Store address of DWORD in rax register (4 byte value, 8 byte pointer)
    push rax                ;Push address of DWORD as the next parameter
    push (msg_end - msg)    ;Push number of bytes to write (3rd parameter)
    push msg                ;Push pointer to constant data bytes (2nd parameter)
    push rbx                ;Push value stored in 'rbx' (The HANDLE) into the 1st parameter
    call _WriteFile@36      ;Calls WriteFile(hStdOut, msg, msg_len, bytes_written, NULL). @36 signifies 36 bytes of parameters

    push 0                  ;Push 0 onto the stack (parameter)
    call _ExitProcess@4     ;Calls ExitProcess(0) and ends the program. @4 signifies 4 bytes of parameters

                ; ---- << Nothing gets called a beyond this line >> ---- ;

    hlt                     ;I have no clue why you need to halt here
    msg:                    ;Label marking the start of the message
        db "Hey world", 10  ;Actual data, (db = define bytes). "Hey world" and 10 (new-line character)
    msg_end:                ;Label marking the end of the message

To compile, I run the following

nasm -f win64 Main.asm -o Main.bin

and to link I run this and get the following output...

gcc -o Hello.exe Main.o
bin/Main.o:src/Main.asm:(.text+0xa): undefined reference to `_GetStdHandle@4'
bin/Main.o:src/Main.asm:(.text+0x21): undefined reference to `_WriteFile@36'
bin/Main.o:src/Main.asm:(.text+0x28): undefined reference to `_ExitProcess@4'
collect2.exe: error: ld returned 1 exit status

As you can see, it can't link to the external functions, what's interesting is that it works perfectly fine when I try to call 'c' functions such as 'puts'

Anyways, if you could help me out here and point out any errors in my code I would greatly appreciate it.

Spice
  • 216
  • 1
  • 3
  • 12
  • Don't you need to link against kernel32? – Michael Jan 25 '19 at 16:05
  • @Michael Well actually, I forgot to mention in the code but gcc already does by default unless you explicitly write -nodefaultlib – Spice Jan 25 '19 at 16:06
  • 3
    Those are the 32 bit functions. 64 bit functions don't have the `@` suffix and the calling convention is different. – Jester Jan 25 '19 at 16:09
  • https://www.davidgrantham.com/nasm-console64/ has a win64 example – Michael Jan 25 '19 at 16:12
  • @Jester I see, so I've read up before on how you need to define 'Shadow Space' for your function call but I wasn't sure how I was supposed to implement that in my program. What if a function requires more than 32 bytes of parameters? Do I have to do something differently? – Spice Jan 25 '19 at 16:45
  • You put the extra arguments on top of the 32 bytes which are simply the space reserved for the first 4 arguments passed in registers. – Jester Jan 25 '19 at 16:48
  • @Michael thank you for the resource, I made the proper modifications to my code and it works now. However I was just wondering why the resource you provided aligns the stack to a multiple of 16 bytes using the 8-byte mask. Does it have to do with efficiency? – Spice Jan 25 '19 at 17:29
  • That is required by the calling convention. Yes, it's for efficiency. Some instructions need 16 byte alignment and it's easier to achieve if you know rsp is aligned already. – Jester Jan 25 '19 at 17:30
  • That said, I believe the linked code is doing it wrong, it's misaligning by 8. Can somebody confirm that? – Jester Jan 25 '19 at 17:38
  • you use *x86* names decoration (_@) when *x64* not have it. of course no any say`_ExitProcess@4` in any *x64* lib. you need `extern __imp_ExitProcess:QWORD` declaration and `call __imp_ExitProcess` and so on. the you wrong pass arguments to api. not use `push` this is for *x86*. first 4 args in registers (*rcx,rdx,r8,r9*) must be. – RbMm Jan 25 '19 at 23:11
  • @Jester - [linked code](https://www.davidgrantham.com/nasm-console64/) correct align *rsp*, but not efficient. no sense add/sub *rsp* before/after every api call. this need do once at function begin (sub) and at function end(add). also not need `and RSP, 0FFFFFFFFFFFFFFF0h`. we know that at this point `rsp = 16*n+8`. so we need `sub rsp,40+m*16` – RbMm Jan 25 '19 at 23:17
  • Yeah that `and` was worrying me, I never know if rsp is supposed to be aligned before or after the return address. – Jester Jan 25 '19 at 23:20
  • If you're looking for more elegant way how to invoke Windows function and not bother with parameter passing and stack alignment, you may like this example of HelloWorld program for 32bit and 64bit MS Windows: https://euroassembler.eu/eadoc/#HelloWorld – vitsoft Jan 26 '19 at 09:12
  • *rsp* must be aligned on `16*n` before *call* instruction. so at the function begin we can assume `rsp == 16*n + 8`. – RbMm Jan 26 '19 at 15:38

0 Answers0