38

i would like to know the exact difference between Commit Size (visible in the Task Manager) and Virtual Size (visible in SysInternals' Process Explorer).

The Virtual Size parameter in Process Explorer looks like a more accurate indicator of Total Virtual Memory usage by a process. However the Commit Size is always smaller than the Virtual Size and I guess it does not include all virtual memory in use by the process. I would like somebody to explain what is exactly included in these parameters.

David Ferenczy Rogožan
  • 23,966
  • 9
  • 79
  • 68
Faris Zacina
  • 14,056
  • 7
  • 62
  • 75

3 Answers3

67

Memory can be reserved, committed, first accessed, and be part of the working set. When memory is reserved, a portion of address space is set aside, nothing else happens.

When memory is committed, the operating system guarantees that the corresponding pages could in principle exist either in physical RAM or on the page file. In other words, it counts toward its hard limit of total available pages on the system, and it formally creates pages. That is, it creates pages and pretends that they exist (when in reality they don't exist yet).

When memory is accessed for the first time, the pages that formally exist are created so they truly exist. Either a zero page is supplied to the process, or data is read into a page from a mapping. The page is moved into the working set of the process (but will not necessarily remain in there forever).

Every running process has a number of pages which are factually and logically in RAM, that is these pages exist, and they exist "officially", too. This is the process' working set.
Further, every running process has pages that are factually in RAM, but do not officially exist in RAM any more. They may be on what's called the "standby list" or part of the buffer cache, or something different. When these are accessed, the OS may simply move them into the working set again.
Lastly, every process has pages that are not in RAM at all (either on swap or they don't exist yet).

Virtual size comprises the size of all pages that the process has reserved.

Commit size only comprises pages that have been committed.

That is, in layman terms, "virtual size" is pretty much your own problem, and only limited by the size of your address space, whereas "commit size" is everybody's problem since it consumes a global limited resource (RAM plus swap). It therefore affects other processes.

Damon
  • 67,688
  • 20
  • 135
  • 185
  • I have noticed that the Virtual Size in one of my apps is rising constantly up to 2GB in a 32bit app and not getting -unreserved- (or whatever other term is used), but the Commit memory is getting uncommited in time and the amount is much lower. is that still considered a Virtual Memory leak? or to rephrase is the Commit size or Virtual size a better indicator of memory leaks? – Faris Zacina Mar 04 '14 at 14:46
  • 1
    Hard to tell. Both _can be_ memory leaks or indicators of such, but they're different things. When your virtual size reaches 2GB (or earlier, due to fragmentation, or later with /3G), you'll eventually run out of address space, so further allocations will get "out of memory" errors although there may in principle be enough RAM available. Generally, "constantly rising and never going down again" is no good, but if _commit_ goes down again, this may just be the heap manager keeping memory reserved (but properly un-commited), which would be OK. Do you encounter actual out of memory conditions? – Damon Mar 04 '14 at 15:34
  • The fragmentation makes sense. I have noticed that most of the reserved memory is on the heap. Yes i do encounter out of memory conditions in the app when the reserved memory approaches 2GB even if the commited memory is 300-400MB or similar. The reserved virtual memory just goes up to a point of crash. – Faris Zacina Mar 04 '14 at 15:36
  • 1
    Problem with fragmentation is that you may make it hard (impossible) for the allocator to provide memory, even though there is enough of it. For example, the typical pattern that `std::vector` uses to grow is _a nightmare_ for the allocator: Allocate a bigger block, copy, then free the smaller block. Then allocate another yet bigger block, copy, and free the 2nd block. It's impossible to recycle the freed memory in a useful way, since the block sizes never fit the sizes you ask for (luckily this is a rare event since `vector` grows geometrically, but you might use similar allocation patterns). – Damon Mar 04 '14 at 15:44
  • 1
    So if you encounter crashes in these conditions when there should actually be enough memory, then yes, it is likely that you're running into fragmentation. Try to be careful in which patterns you allocate/free, reserve early instead of growing dynamically, and if applicable allocate larger pools of objects (pooling objects that you reuse over and over again can work wonders). Also be aware that the allocator has minimum sizes to cope with (usually 16 bytes, since it aligns to 8 bytes and needs to store bookkeeping info), so allocating many very small objects is _poison_ to the allocator, too. – Damon Mar 04 '14 at 15:46
  • 2
    Don't confuse `Commit Size` (reported by Task Manager) with `Committed Pages`. The `Commit Size` is the amount of memory which is not backed by memory mapped files (such as binary code pages). `Committed Pages` are a segment of Virtual Memory which includes all addressable pages (as opposed to Free or Reserved). – josh poley Mar 04 '14 at 16:58
  • @joshpoley: Please refer to http://windows.microsoft.com/en-us/windows/what-task-manager-memory-columns-mean : _"Memory - Commit Size: Amount of virtual memory that's reserved for use by a process"_. Here, the word "reserved" means "reserved from the total pool of available pages in the system (physical RAM + pagefile), not anything different. _Some_ mapped pages (not all!) are not captured by that number, that's correct (but irrelevant). Generally, "backed by memory mapped files" is a wrong wording, only read-only mappings do not count towards commit (read-write and copy-on-write do). [...] – Damon Mar 04 '14 at 17:16
  • 1
    [...] Fully position-independent code, as you've pointed out, is one example of mappings that typically don't count towards commit (but other sections may). Also, you are right that the number shown in TaskManager does differ from the number of pages by a factor equal to the page size (but that's only a presentation detail). Still, this memory is certainly committed in complete pages, and it's the same figure, only the number shown in that particular tool is bigger. It simply isn't possible to commit anything but complete pages (that's not 100% correct either, since alloc granularity is 64k). – Damon Mar 04 '14 at 17:20
  • @Damon I see in Task Manager (Win7 sp1) for some processes that Commit Size is smaller, sometimes much smaller than Working Set. For example for taskmgr.exe it is 4188 vs 7388 KB. For vmware-vmx it is 46 vs 301 MB. Obviously Working Set is not part of Commit Size nor Commit Size is "amount of virtual memory that's reserved for use by a process". As Hans Passant wrote for Commit Size - "amount of space reserved in the paging file for the process" - that can be. – ggurov Jan 18 '18 at 11:45
  • @Damon I've made a diagram illustrating your post - is it okay if I edit your post to add it? – Dai May 10 '22 at 10:52
9

Commit size is the amount of space reserved in the paging file for the process. Used when its pages need to be swapped out to make room in RAM for other processes.

And yes, the virtual memory size will be larger since it includes the pages that are not backed by the paging file. At least the code and resource sections in the EXE and non-relocated DLLs. When necessary, those pages can simply be discarded and reloaded from the file. All the operating system DLLs fit this category. Additionally any memory-mapped files used by the process. SysInternals' VMMap utility can give insight.

Hans Passant
  • 922,412
  • 146
  • 1,693
  • 2,536
2

Virtual size is the size of the process's virtual address space.

When allocating memory, you can request an address range without actually committing memory behind it. Those address do not point to physical memory until a second allocation call is used to commit them.

Read more on VirtualAlloc. Linux has the mmap function that does the same thing.

egur
  • 7,830
  • 2
  • 27
  • 47