1

Sometimes code that use heap allocation functions throw this error: munmap_chunk(): invalid pointer It appears by context munmap_chunk() is a function, and these errors are this function throwing them. What is this function and what does this function do?

My question is rather simple, so there is no reason to provide a lot of extra information: What is munmap_chunk()?

  • Need much more details. If you get that "a lot of times", either your OS installation or your use account settings are broken, or a lot of your programs are broken. – hyde Aug 12 '19 at 18:58
  • Possible duplicate of [What is the functionality of munmap, mmap](https://stackoverflow.com/questions/41480184/what-is-the-functionality-of-munmap-mmap) – TylerH Aug 12 '19 at 18:59
  • 1
    @hyde No details needed, it's a very specific question about whether munmap_chunk is a function/syscall/somethingelse, and what does it do. – Greg Schmit Aug 12 '19 at 19:02
  • 1
    It is a function. Look at the [source](https://github.com/lattera/glibc/blob/master/malloc/malloc.c). –  Aug 12 '19 at 19:04
  • @GregSchmit Ok, looking at the answer, the question isn't that unclear. Still, did you Google "glibc munmap_chunk" first? Explaining prior research in a question is generally appreciated in SO questions. – hyde Aug 12 '19 at 19:55
  • @hyde In my opinion, this _is_ an example of a typical "unclear-what-you-are-asking" or "too-broad" question on SO. I just think that many (or maybe most) of such questions _can_ be answered. The problem is that they require a specific person with a very specific knowledge. I just randomly happen to know some things on the topic, so I provided my answer. But this is a discussion for meta. –  Aug 12 '19 at 20:22
  • 2
    @hyde When I used Google, I got a ton of fixes for "invalid pointer" bugs, but nothing about `munmap_chunk` itself. I agree that he could've explained his research more, but he did explain the context of where you usually see `munmap_chunk`. – Greg Schmit Aug 12 '19 at 20:27
  • @StaceyGirl I don't think it's either unclear or too broad, it's a specific question about a single thing called `munmap_chunk` and the context is that you always see it coming up with invalid pointers and so people always fix the underlying problem, but no one explains what's actually going on. I've been curious in the past, and Google does not help unless you already know to search for the source in glibc and then you're also good at comprehending low-level library code. – Greg Schmit Aug 12 '19 at 20:34
  • @GregSchmit It could be classified as "too broad" since there are many ways to explain this. Again, I am not voicing my opinion, just talking about how SO handles questions in general. It is rather dangerous to ask questions like this here - it either gets answered quickly or gets closed. –  Aug 12 '19 at 20:37
  • @StaceyGirl I understand, hence why I made all my comments arguing the relevance and appropriateness, because I could feel the oncoming wave of "this question seems hard and I don't know the answer, therefore I'm going to pretend that it is unclear or redirect you to a similar but unrelated question." Luckily, you, a subject matter expert, were here to answer it. – Greg Schmit Aug 12 '19 at 20:41
  • @GregSchmit Yep, that "When I Google, I get..." is exactly what would make this question a lot better (and less likely to get close and down votes). – hyde Aug 12 '19 at 20:59

1 Answers1

3

Two simple functions malloc and free must solve contradicting problems:

  • Memory must be returned to the kernel as fast as possible.
  • Memory allocations must be fast, so memory cannot not be returned to the kernel immediately and should be reused. Coupled with memory fragmentation, this makes it impossible to return memory to the kernel (glibc doesn't do this at all).

To solve both, allocators (at least glibc, musl, windows) have a threshold for a size of an allocated object: allocations greater than that size are done by direct call to mmap/VirtualAlloc while freeing such chunks goes directly to munmap/VirtualFree.

This makes it possible to keep malloc fast for small allocations, while keeping it memory-efficient for large ones (as it can free large chunks of memory immediately).

The threshold value is usually about 128 KB and larger (see DEFAULT_MMAP_THRESHOLD_MIN in glibc, MMAP_THRESHOLD in musl).

Function munmap_chunk in glibc is responsible for that and is usually called like this:

if (chunk_is_mmapped(p)) {
    munmap_chunk(p);
    return;
}

You will see munmap_chunk(): invalid pointer message when passing a pointer to a chunk of memory that looks like mmap-allocated chunk, but upon investigation has invalid size, which means there is a memory corruption or free misusage.

Aside from this, it is also responsible of keeping track of a number of allocated through mmap chunks to workaround VMA merging issue on Linux kernel issues.

  • This threshold is also something that make it efficient to implement custom pool allocators on top of the system one: such allocator usually work in large chunks and thus get their memory from `mmap` through `malloc`, making it possible to actually return the memory once your are done with it. –  Aug 12 '19 at 19:37
  • To see how memory can be returned to the kernel despite memory fragmentation, you should read about [`MADV_FREE`](http://man7.org/linux/man-pages/man2/madvise.2.html). –  Aug 12 '19 at 19:41