20

Just the question stated, how can I use mmap() to allocate a memory in heap? This is my only option because malloc() is not a reentrant function.

trincot
  • 317,000
  • 35
  • 244
  • 286
domlao
  • 15,663
  • 34
  • 95
  • 134

2 Answers2

32

Why do you need reentrancy? The only time it's needed is for calling a function from a signal handler; otherwise, thread-safety is just as good. Both malloc and mmap are thread-safe. Neither is async-signal-safe per POSIX. In practice, mmap probably works fine from a signal handler, but the whole idea of allocating memory from a signal handler is a very bad idea.

If you want to use mmap to allocate anonymous memory, you can use MAP_ANON/MAP_ANONYMOUS.

p = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

At one point this was not strictly portable, and systems differed in which spelling they supported so you should write preprocessor conditionals to use whichever is available, but POSIX has since adopted both spellings as standard.

The traditional but ugly version, from long ago before MAP_ANON was a thing, is:

int fd = open("/dev/zero", O_RDWR);
p = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
close(fd);

Note that MAP_FAILED, not NULL, is the code for failure.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • 1
    Actually I need to allocate a memory in the signal handler. So basically I need a reentrant implementation for this. I saw the malloc_r and free_r which was created for reentrancy but only available for my compiler. Thanks – domlao Jan 24 '11 at 06:30
  • 4
    Then there's no conformant way to allocate memory, but `mmap` will "probably" work. It would be a lot better to fix the design that makes it necessary to allocate memory from a signal handler. Normally a signal handler should either do nothing or just set a single flag variable or write a byte to a pipe. – R.. GitHub STOP HELPING ICE Jan 24 '11 at 06:48
  • 3
    By the way, since "do nothing" probably was not clear, a "do nothing" signal handler is useful with the `SA_RESTART` flag omitted to interrupt syscalls. Setting a do-nothing signal handler to interrupt syscalls and and using `pthread_kill` to send the signal to a particular thread is a way to "roll your own" thread cancellation without the unfixable resource leak issues `pthread_cancel` leads to. It can also be useful with just a single thread if you set a timer/alarm to generate the signal, to set timeouts for syscalls. – R.. GitHub STOP HELPING ICE Jan 24 '11 at 07:05
  • 5
    The most portable version is probably not to open `/dev/zero` but to use `shm_open` instead, which internally does about the same thing but doesn't require your file system with special files to be up. – Jens Gustedt Jan 24 '11 at 07:56
  • 3
    Is `MAP_PRIVATE` valid with shared memory obtained via `shm_open`? I suppose so, since I couldn't find anywhere it's explicitly prohibited, but this seems counter-intuitive. – R.. GitHub STOP HELPING ICE Jan 24 '11 at 16:08
  • 1
    @R: And for the deallocation? Do I need to call the munmap? like this `munmap( p, 0 ) ? – domlao Jan 25 '11 at 04:12
  • 3
    @sasayins: Not like that. You must pass the length of the mapping to `munmap`; you can't just pass 0. – R.. GitHub STOP HELPING ICE Jan 25 '11 at 04:20
  • 1
    oh gee, I guess I need to re-implement. because what I did was I created a malloc and free hooks, I used mmap to __wrap_malloc and I used munmap to __wrap_free. – domlao Jan 25 '11 at 04:27
  • 4
    Just store the size at the beginning of the `mmap`-allocated block, and return a pointer to the byte just after where the size is stored. Then freeing the block is as easy as backing up to read the size and passing the new pointer and size to `munmap`. – R.. GitHub STOP HELPING ICE Jan 25 '11 at 04:29
  • Sorry for the late reaction: I found your question by google, exactly because I want to find a way to allocate dynamical memory from a signal handler. – peterh Sep 30 '18 at 21:44
  • 1
    FYI the resolution of [Austin Group issue 850](https://www.austingroupbugs.net/view.php?id=850) standardized `MAP_ANON` (and `MAP_ANONYMOUS` alias) for future edition of POSIX. – R.. GitHub STOP HELPING ICE Mar 11 '20 at 01:54
  • Accessing `/dev/` can be problematic if you're writing library code which gets used in a chroot environment. [This](https://lo.calho.st/posts/black-magic-buffer/) page suggests `syscall(__NR_memfd_create, name, flags)` instead, but I don't know how portable that is and I came here looking for an alternative (and `-1` will do, thanks!). Though I see in comments `shm_open()`, too. Will investigate. – sh1 Aug 22 '23 at 16:51
  • @sh1: Yes, but that's already solved by `MAP_ANON` being accepted into the standard and already universal anyway. memfd on the other hand is Linux-specific and creates a shared memory object that persists until there are no references to it, which is NOT what's wanted here. – R.. GitHub STOP HELPING ICE Aug 22 '23 at 20:39
  • @R..GitHubSTOPHELPINGICE Good to know. This is, after all, 12 years on. Does the answer need an update regarding which method is most portable? – sh1 Aug 23 '23 at 16:40
  • @sh1: Good idea. I forgot folks might not read all the comments. Does my edit look good? – R.. GitHub STOP HELPING ICE Aug 23 '23 at 20:52
  • Yeah, looks good to me. – sh1 Aug 24 '23 at 21:45
10

Make a simple slab allocator

Although allocating memory in a signal handler1 does seem like something best avoided, it certainly can be done.

No, you can't directly use malloc(). If you want it to be in the heap then mmap won't work either.

My suggestion is that you make a special-purpose slab allocator based on malloc.

Decide exactly what size of object you want and preallocate some number of them. Allocate them initially with malloc() and save them for concurrent use later. There are intrinsically reentrant queue-and-un-queue functions that you can use to obtain and release these blocks. If they only need to be managed from the signal handler then even that isn't necessary.

Problem solved!

1. And if you are not doing that then it seems like you have an embedded system or could just use malloc().

DigitalRoss
  • 143,651
  • 25
  • 248
  • 329