3

I am generating tons of small fragments of executable machine code dynamically, all of them with unpredictable sizes and lifetimes. Currently I am using a naive executable mmap preallocated region, but this is only provisional due to the eventual memory exhaustion. I did not implemented any memory reclamation mechanism and I do not want to: this is a hard problem (hard to debug bugs, fragmentation, multi-threading, etc.).

What I need is a library that implements the malloc/free protocol but with the executable bit enabled.

Is there any malloc implementation that offers this option (can be a compile-time flag)?

Further details to answer comments:

  • My current platform is Linux on x86_64, but Windows and ARM 32/64 support will be welcome (but not a must).
  • Memory never will be shared with different processes, but the allocator must be multi-threading aware and, if possible, scalable in this scenario.
user3368561
  • 779
  • 6
  • 18
  • it is system dependent. What is the system/arch? – bolov Aug 02 '18 at 11:02
  • Is this possible to use the same memory map for multiple executable in the same time ? – Stargateur Aug 02 '18 at 11:05
  • @bolov Question updated to answer you. – user3368561 Aug 02 '18 at 11:11
  • of interest: https://stackoverflow.com/questions/3125756/allocate-executable-ram-in-c-on-linux , https://stackoverflow.com/questions/40936534/how-to-alloc-a-executable-memory-buffer – bolov Aug 02 '18 at 11:13
  • @Stargateur Do you mean if this `mmap` is shared or private? It's private and anonymous if it matters. – user3368561 Aug 02 '18 at 11:13
  • Do you need to be able to malloc() in one thread and free() the result in another? Are you willing to set a fixed maximum memory size for the entire program throughout its lifetime? – John Zwinck Aug 02 '18 at 11:32
  • @JohnZwinck If possible, I want to be able to `malloc()` on one thread and `free()` in another, but it isn't a must. Memory usage is currently unbounded, but it can be bounded if required. Anything better than my current "solution" (an unsafe hack that crashes the program when buffer is full) will be welcome. – user3368561 Aug 02 '18 at 11:48

1 Answers1

3

The easiest way is to use a pool (arena) allocator. For example this one: https://github.com/philip-wernersbach/memory-pool-allocator . The implementation calls malloc() once during initialization, but on most systems if you call malloc() with a large enough size, it will use mmap(). So you could modify this library to call mprotect() or similar to make the pages executable, and maybe even replace the malloc() call with mmap() directly (because the threshold at which malloc() calls mmap() is configurable, so you may not want to rely on it).

John Zwinck
  • 239,568
  • 38
  • 324
  • 436