malloc and free x86 NASM + compiling

Question

I have been teaching myself x86 assembly and have been looking at doing basic malloc() and free() calls. I have spent quite a bit of time searching but most examples are for 64-bit or only show the malloc call without the free, etc. I even wrote this in c, then compiled and disassembled it, which helped but gcc adds a lot of other instructions.

Here is a basic example I made of what I was able to figure out, please let me know if this is correct or if there is anything else I should be doing:

global _start
; glibc stuff
extern _malloc, _free

section .data
  err: db "malloc failed!", 10, 0
    .len: equ $ - err

section .bss
  mptr resd 1   ;pointer to begining of malloc'd memory

section .text
_start:

  push 20       ;allocate 20 bytes
  call _malloc  ;call malloc
  add esp, 4    ;clean pushed imm

  test eax, eax ;check for malloc error
  jz merror

  mov [mptr], eax ;store address

  mov byte [eax], 0
  mov byte [eax + 1], 1

  push mptr     ;push address
  call _free    ;call free
  add esp, 4    ;clean push

exit:
  mov eax, 0x1
  int 80h

merror:
  mov eax, 0x4
  mov ebx, 0x1
  mov ecx, err
  mov edx, err.len
  int 80h
  jmp exit

The second part to my question is compiling it. From what I was able to find I need to link /lib/ld-linux.so.2. So in my makefile I have the following but it errors out:

mem: mem.asm
    nasm -f elf mem.asm
    ld -melf_i386 -lc -I /lib/ld-linux.so.2 mem.o -o mem

This is the error I get when trying to compile:

As I said I am a noob at x86 so if you also have any comments for better ways to do things I would appreciate those too! :)

UPDATE :

So I went ahead and used gcc and got that to work (without and errors at least):

mem: mem.asm
    nasm -f elf mem.asm
    gcc -m32 mem.o -o mem

However when I went to run it it crashed big time:

I am clearly doing something wrong with free but as I mentioned, I wasn't positive about my use of malloc and free since I couldn't find any solid examples. Any clues?

I believe you have to link the C runtime (`/lib/i386-linux-gnu/libc.so` or something?) Also if you start from `_start` (kind of a pun intended) you need to initialize the C runtime. In a nutshell, if you need the C runtime, use GCC to link your object files. — Margaret Bloom, Aug 26 '17 at 20:18
And start from `main` (so let the clib to initialize). The difference is, that you are not calling some magic `malloc` which will fully allocate new memory, but ordinary C function called `malloc`. It's not full memory manager, but only sub-part of it, which will give you some chunk of memory from the internal heap already allocated by the underlying memory manager code. Thus it's very likely that implementation already expects to be initialized. If you provide `main`, then the C is already initialized, when it jumps to `main`. — Ped7g, Aug 26 '17 at 20:23
Actually your problem, besides that init thing, and that you can probably use `gcc` wrapper to link it with simpler line, as you want clib any way, is in linux the c symbols don't need underscore IIRC, i.e. `extern malloc` `call malloc`. I have found some old example of calling C functions from asm here: https://stackoverflow.com/questions/24991944/linking-c-with-nasm (looks like your `ld` invocation is actually maybe OK) — Ped7g, Aug 26 '17 at 20:31
Note also: By far, the most common way to use asm on Linux is GCC's "inline asm" within C source files. — o11c, Aug 26 '17 at 20:48
@o11c it may be common, but it is lot more tricky. If the OP is just learning the x86 assembly, it's much better for him to use separate asm files, so he doesn't need to learn two things at the same time (x86 assembly + gcc inline asm rules, which are far from trivial). And inline asm is often a bit abused, in many cases it would do well with pure C (a bit adjusted in favour of compiler result), or with intrinsics only, writing inline asm should be sort of "last resort" solution when everything else fails. — Ped7g, Aug 26 '17 at 20:57
So i went ahead and used gcc to link. Got that working. Now its just my use of malloc and free that need help. Thanks for all the help so far! *SEE UPDATE — , Aug 26 '17 at 21:03
`push mptr ;push address` will push address, but address of `mptr`, not value stored in memory there. Try `push dword [mptr]`. And you need debugger, if you would use one, you would see the stack ahead of `free` call contains different address than the one returned by `malloc`... Also update the asm in Q? Or you didn't use `main` yet? Then I'm surprised the `malloc` returns something usable, maybe it has lazy init and detects it's being called first time and does init everything properly. Even `strace` would probably reveal discrepancy in argument values. — Ped7g, Aug 26 '17 at 21:15
You seem to be following a tutorial for Windows. Note that on Linux, symbols are not decorated with underscores! Also, you should really not use the C library without intitializing it first by having the C compiler insert the appropriate start stub for you. — fuz, Aug 26 '17 at 22:28
@fuz. How would i go about getting the compiler to do that? I am a newbie just teaching myself so anything i can do to get better im all ears — , Aug 27 '17 at 00:38
The easiest and safest thing is to write a `main()` function and let gcc include the crt start files that do everything they normally would before calling a `main()` produced by a C compiler. If you insist on writing your own `_start`, see https://stackoverflow.com/questions/36861903/assembling-32-bit-binaries-on-a-64-bit-system-gnu-toolchain/36901649#comment61368106_36901649 for which glibc init functions you have to call (from a static binary). Or use a different libc like MUSL that doesn't need to be initialized: https://stackoverflow.com/a/35210404/224132 — Peter Cordes, Aug 27 '17 at 01:16
@fuz: Good point, but dynamically linked binaries don't need to call the glibc init functions from `_start`, because the magic of dynamic linking runs them. (i.e. they're called by `ld.so` before it jumps to your `_start` code.) But for a static binary, see the previous comment. (@ hb: use `file a.out` to check for sure that it's dynamically linked. Oh, you're already defining `main()` not `_start`, and you aren't using `gcc -nostartfiles` or anything. You're totally fine here. But those links might help you understand some basics of how Linux processes start up.) — Peter Cordes, Aug 27 '17 at 01:18
*I have updated the asm in the original question to reflect the correct answer.* Please don't do that. Write up the answers as an Answer post (click "answer your own question". Writing up the solution as a proper answer is encouraged when you solve a problem yourself, if there's any future value in the question for other searchers who might have the same problem). Your edit has turned this question into a non-question, which is not how we do things here at SO. I rolled back your edit; you can still get the source from the edit-history if you want to copy that to an answer. — Peter Cordes, Aug 27 '17 at 01:23

score 3 · Accepted Answer · answered Aug 27 '17 at 13:08

Thanks to everyone for the help! So first I was able to get the linking errors fixed by using gcc to link instead of ld:

mem: mem.asm
    nasm -f elf mem.asm
    gcc -m32 mem.o -o mem

In order to get that to work I needed to change the names of the functions from _malloc and _free to malloc and free. I also had to change the standard global _start to global main in order to get gcc happy. This let it compile and link without errors but as you saw in the update the program crashed horribly when it came time to free the memory.

This was because I was pushing the wrong address to the stack. I initially had the instruction push mptr but that was pushing the address of mptr to the stack rather than the address it was pointing to, hence the error. A simple update to the instruction in order to push the correct address to the stack allowed my simple program to run without errors:

push dword [mptr]

The final result:

global main
; glibc stuff
extern malloc, free

section .data
  err: db "malloc failed!", 10, 0
    .len: equ $ - err

section .bss
  mptr resd 1   ;pointer to begining of malloc'd memory

section .text
main:

  push 20       ;allocate 20 bytes
  call malloc   ;call malloc
  add esp, 4    ;clean pushed imm

  test eax, eax ;check for malloc error
  jz merror

  mov [mptr], eax ;store address

  mov byte [eax], 0     ;store 0 at index 0
  mov byte [eax + 1], 1 ;store 1 at index 1

  push dword [mptr]     ;push address
  call free             ;call free
  add esp, 4            ;clean push

exit:
  mov eax, 0x1
  int 80h

merror:
  mov eax, 0x4
  mov ebx, 0x1
  mov ecx, err
  mov edx, err.len
  int 80h
  jmp exit

Thanks again to everyone for the help and thanks Peter Cordes for giving me a chance to provide a proper answer! :)

I'm sure i'll be back with more noob x86 questions as my journey continues!

malloc and free x86 NASM + compiling

1 Answers1