8

First of all, let me say that I very well know how VirtualAlloc() works and how virtual memory facilities of the modern operating systems, in general, work.

Said that, using the MAP_RESERVE flag of the VirtualAlloc() Windows API, without using also MEM_COMMIT, has some practical use today?

I mean, when I call VirtualAlloc() with MEM_RESERVE|MEM_COMMMIT I both reserve and commit pages, but I know that the operating system will really allocate pages only when I try to write into them. This optimization happens on practically all the modern operating systems.

So, with this optimization in mind, if I call VirtualAlloc() with MEM_RESERVE and then I call it several times with MEM_COMMIT to commit pages, isn't the same result of calling VirtualAlloc() only once, specifying MEM_RESERVE|MEM_COMMIT ?

Since specifying only MEM_RESERVE will only reserve pages boundary addresses, without committing real pages, but then MEM_RESERVE|MEM_COMMIT will reserve+commit only pages I write to, isn't using MEM_RESERVE alone, a waste of time today? With only 1 call to VirtualAlloc() with MEM_RESERVE|MEM_COMMIT I can basically obtain the same result of calling VirtualAlloc() 1 time with MEM_RESERVE and calling it several N times with MEM_COMMIT.

As a proof of what I am saying, I have noticed that the MEM_RESERVE facility doesn't exist at all in Unices/POSIX systems which use the mmap(2) syscall. Also there you can "commit" a big chunk of pages calling mmap(2) once, and then the pages will be really allocated only when you write into them, all optimized by the operating system.

So, is using MEM_RESERVE alone just an old days thing useful only when memory pages were a precious resource, and so today is useless? Or, using this flag alone (and then calling VirtualAlloc() N times with MEM_COMMIT) still has some practical use that I am missing?

Marco Pagliaricci
  • 1,366
  • 17
  • 31
  • 2
    See http://stackoverflow.com/q/10494588/613130 The difference is in the handling of the swap file (if the response there is correct) – xanatos Jul 28 '15 at 09:17
  • On a 32 bit system with limited address space I can see that `MEM_RESERVE` by itself would still be useful. On a 64 bit system there's probably not much point. – Jonathan Potter Jul 28 '15 at 09:30

2 Answers2

10

When you commit memory, the memory manager doesn't immediately allocate real pages for you, but it does count them towards the total number available. The memory manager will never commit more pages than actually exist, and you are guaranteed that when you access a committed page, the memory is there.

MEM_RESERVE's purpose is to allocate virtual memory addresses. You can reserve as much address space as you want, up to many terabytes on 64-bit systems, even if you don't have many terabytes of actual space.

By default, Linux doesn't limit allocations to the amount of actual space available. This "feature" is called overcommit. That means that on Linux there are no separate reserve and commit steps. If you want to reserve address space on Linux, another question recommends mmapping an area with no permissions..

user253751
  • 57,427
  • 7
  • 48
  • 90
  • 1
    Yeah, as I've guessed it was related to the swap space,the Linux overcommit facility, etc. The answer in http://stackoverflow.com/questions/10494588/benefits-of-reserving-vs-committingreserving-memory-using-virtualalloc-on-larg explains it very clearly. I knew that `MEM_RESERVE` basically only writes a new node in a tree structure, representing your reserved address ranges. Then when you commit, the OS will really face reality and realize if it can commit those pages, checking the memory, swap file etc. That's very interesting the method of `mmap(2)` and `PROT_NONE`+`MAP_NORESERVE` -- thanks. – Marco Pagliaricci Jul 28 '15 at 09:50
3

In addition to the previous answer here is a quote from Microsoft docs:

As an alternative to dynamic allocation, the process can simply commit the entire region instead of only reserving it. Both methods result in the same physical memory usage because committed pages do not consume any physical storage until they are first accessed. The advantage of dynamic allocation is that it minimizes the total number of committed pages on the system. For very large allocations, pre-committing an entire allocation can cause the system to run out of committable pages, resulting in virtual memory allocation failures.

Dmitry Shesterkin
  • 308
  • 3
  • 8