Is merging pages allowed in mmap?

Question

In short I want to resize memory but have the old memory in the middle of the new.

So what I did was use mmap for the initial size (p1), mmap at an address before p1 to pretend I made the memory bigger, then treat the new pointer as if I created it with a single mmap (p3, mremap). The code seems to work but I'm not sure if this is what I should be doing. If it isn't how should I create more memory and have the old/current memory be in the middle of it?

#include <sys/mman.h>
#include <cstdio>
#include <cstring>
#include <cassert>
#include <cerrno>
int main()
{
    auto p1 = (char*) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (!p1 || p1==(char*)-1)
        return -1;

    auto p1_32 = (int *)p1;
    memset(p1, 0, 4096);
    p1_32[5] = 1035;

    auto p2 = mmap(p1-4096, 4096, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0);
    if (!p2 || p2==(void*)-1)
        return -2;

    auto p2_32 = (int *)p2;
    memset(p2, 0, 4096);
    p2_32[5] = 5301;

    assert(p2_32[1024+5] == 1035);

    auto p3 = mremap(p2, 4096*2, 4096*4, MREMAP_MAYMOVE);
    if (p3==(char*)-1)
    {
        printf("Errno %d\n", errno);
        return -2;
    }
    auto p3_32 = (int*)p3;

    assert(p3_32[5] == 5301);
    assert(p3_32[1024+5] == 1035);

    printf("Is this correct?\n");
    return 0;
}

Few notes: 1) you don't have to zero-out memory returned with `mmap` - it is already zero-initialized (otherwise it could leak sensitive information from another process). 2) I think you should tag this with `linux` since `mremap` is not available on some OSes. — , Mar 02 '20 at 11:16
@StaceyGirl I do not think that the memory is really guaranteed to be zero-filled. The system takes measures to avoid leaking sensitive information, this is correct, but this does not necessarily mean that the pages contain zeroes. Perhaps previously mapped pages of the same process can be reused — Ctx, Mar 02 '20 at 11:20
@StaceyGirl But the `memset()` calls will force the actual assignment of physical memory to the virtual address. Those mappings are not created until the memory is actually accessed, and deferring those mappings can cause serious performance issues for time-critical processing if that processing stalls because the kernel has to actually create the memory pages. — Andrew Henle, Mar 02 '20 at 11:22
@Ctx In this case spec requires memory to be filled with zeroes (if man pages can be called spec). There is `MAP_UNINITIALIZED ` for embedded system though. — , Mar 02 '20 at 11:22
@AndrewHenle I don't think this matters in this case. If page faults need to be avoided, there is `MAP_POPULATE` specifically for this - which will do this faster. — , Mar 02 '20 at 11:23
@StaceyGirl Hm, this manpage apparently does not specify that the pages are zeroed out: https://linux.die.net/man/3/mmap — Ctx, Mar 02 '20 at 11:24
@AndrewHenle But `mlock()` is probably more appropriate for that purpose. — Ctx, Mar 02 '20 at 11:26
Note that the memory just before and just after your mapping might already be used. — user253751, Mar 02 '20 at 11:32
@AndrewHenle But `memset()` does not guarantee that the pages are still in memory in the next microsecond, so I think, it wouldn't be more than a hack to use it for time-critical processing — Ctx, Mar 02 '20 at 11:46
@Ctx Time-critical processing is properly done on a system that has known-to-be-sufficient resources by processes that don't oversubscribe those resources. — Andrew Henle, Mar 02 '20 at 11:55
@Ctx: That man page doesn't even describe mapping anonymous memory, which is the only time zero-filling comes up. Otherwise, you're mapping *something*, which means you wouldn't get zeroes, you'd get whatever was in that file, object, what-have-you. On Linux, `mmap` is implemented as copy-on-write mappings of the zero page. — ShadowRanger, Mar 02 '20 at 11:56
@AndrewHenle Yes, but you have to tell the system, how to use these resources, specifically, which pages are to reside in memory and which are not. `mlock()` and `mlockall()` is used for this task. — Ctx, Mar 02 '20 at 11:57
@ShadowRanger Correct, this is how it is currently implemented. But this does not mean, that it may not change in the future or differ on other systems, so it is probably bad to rely on the specific current behaviour. — Ctx, Mar 02 '20 at 12:09

score 3 · Accepted Answer · 2020-03-02T20:01:47.027

As described here

The munmap() function shall remove any mappings for those entire pages containing any part of the address space of the process starting at addr and continuing for len bytes.

so it is allowed to remove multiple mappings with a single munmap call (as if it was a single mapping).

There is a problem with your code though: how do you know if a page (p2) before your page (p1) is not used? It could be already allocated by other pars of the program (including malloc), by using MAP_FIXED like this you will rewrite (remap) its content:

When MAP_FIXED is set in the flags argument, the implementation is informed that the value of pa shall be addr, exactly. If MAP_FIXED is set, mmap() may return MAP_FAILED and set errno to [EINVAL]. If a MAP_FIXED request is successful, the mapping established by mmap() replaces any previous mappings for the process' pages in the range [pa,pa+len).

So I don't think this trick can be useful in the general case, you should use mremap instead.

As for how this is implemented: Linux does merge sequential private anonymous mappings, so both will be merged into a single vma_struct in the kernel. This "feature" has undesirable side effects such as munmap failing to free memory with ENOMEM. But this is more of an implementation detail, not something you have control over.

I figured I'd have to write more logic to make sure p1 and p2 connect. How would I implement this using mremap? I don't see a way to specify where the old contents should be. — Eric Stotch, Mar 02 '20 at 16:01
I have another idea how to implement this. I could use `mremap` to move it into a larger location, then immediately remap it within itself + mmap the space before it. Although I'm not sure if mremap-ping into itself is legal — Eric Stotch, Mar 02 '20 at 16:08
@EricStotch You can try to a specific address without `MAP_FIXED`, this will act as a hint: a kernel will map data there only if there is no mapping in place. `mremap` with address will do the same thing - it won't work for your case. You can try file mappings instead, but instead of file you can use a pseudo file returned by `memfd_create` - such file allows you to preserve data even after unmapping and remapping it. — , Mar 02 '20 at 16:43

Is merging pages allowed in mmap?

1 Answers1