VMMap - Why is Committed memory 13.4GB for one exe? Working Set is only 1.7GB

Question

So my question is about this one exe that seems to be eating my server's RAM up. It is a Windows Server 2008 R2. Rebooting doesn't seem to help, as the Commit Size will remain more or less 13GB. This came to my attention because the vendor wants more RAM because of performance issues with the program.

The main thing that worries me is that it doesn't look like the program is actually using all of the RAM it has asked for. I haven't yet seen that Private Working Set memory get above 4GB. I have watched the server while users are complaining about slowness and locking up workstations and it still isn't using all of the committed memory.

Is this a memory leak? What is going on with this exe to cause this and why isn't it using all of its committed RAM when it actually needs it?

Namely I am trying to come up with concrete proof that this is the vendor's problem so the vendor stops trying to blame me.

This is a VM, as a troubleshooting step I went into the VM settings and made sure the memory was allocated properly, it is not overcommitted, and I checked "unlimited" for the memory resource allocation.

Thanks for any advice or any additional troubleshooting I can try!

Answer 1

Processes in Windows don't ever "commit RAM". They commit virtual address space.

The main thing that worries me is that it doesn't look like the program is actually using all of the RAM it has asked for.

Programs don't "ask for RAM"! They ask for committed virtual address space.

What Windows shows as "committed" storage is not RAM. It is a guarantee of availability of virtual memory, should it actually be accessed. If it's ever used the storage may be partly in RAM and partly in the pagefile (assuming you have a pagefile).

The "working set (private)" counter shows approximately how much of the process's committed virtual address space is in RAM. I say "approximately" because part of committed memory, after it's been in the process working set for a while, may be moved to the modified page list. From there it will be quickly written to disk and moved to the standby page list, from which the RAM can be repurposed for something else. (Or if it's brought into the process and not written to, it can be dropped from the working set directly to the standby list.)

These lists are in RAM but they're not in the working set. If the process references a page while it's still on one of those lists, that's a page fault, but it's a soft page fault that's resolved without going to disk. It's just taken from the list and put back in the process working set.

"Committed" allocations of virtual address space will only consume, at most, as much RAM as they have actually accessed. It is not at all uncommon for a program to commit more than it actually uses on any given run... although the ratio is usually not quite this large! "At most" because the OS memory manager may remove long-ago-referenced private pages from the working set in order to free RAM for things that are being accessed more often.

If this program is causing other programs to fail with "low on memory" or "out of memory" errors, this would be an excellent time to enlarge your pagefile. Those error messages have to do with commit limit, not RAM. Enlarging your pagefile will raise the commit limit (hence eliminating the error messages) but since this program is not actually using that much RAM it won't cause any more actual pagefile IO!

The "commit" mechanism simply guarantees that once the OS has told a process "since you have successfully committed this amount of virtual memory, it is guaranteed that the OS have a place to keep it even if you happen to use it all, later in your run." But if the program doesn't actually reference it all, the part it doesn't use doesn't take any significant physical storage anywhere. Not in RAM, not in the pagefile, not anywhere.

Answer 2

Profiling tools: If the program is written in .NET, try Yourkit, ants, dotTrace, etc. If native, debugging tools available through Windows Development Kit such as UMDH are good tools for tracing memory leaks. Although it will only make sense to you if you have debugging symbols for the program and its dependencies, and preferably the source code.

Although memory leaks have different culprits, if the process' memory consumption is not going gradually up, it would seem to me that the way the application is used after it got to such memory footprint doesn't leak memory anymore. Or maybe it's just inefficient. Or maybe it does require 13GB for some reason.

Now if it'll consume memory until it exhausts the resources of the computer of goes beyond the limits of the OS, then it's most likely there's a leak.

I would also recommend trying to find if any operation will trigger the leak. Giving steps to reproduce to your vendor helps.

As for your original question check this SO question.

References

• YourKit: memory profiler
• RedGate: Ants memory profiler
• JetBrains: DotTrace memory profiler
• Microsoft: User-Mode Dump Heap (UMDH)

Answer 3

The main thing that worries me is that it doesn't look like the program is actually using all of the RAM it has asked for. I haven't yet seen that Private Working Set memory get above 4GB.

It must be 32bit in which case WS will never climb higher than 4GB.

Please see my answer here. https://stackoverflow.com/a/44615376/8177258

In Task Manager: Working Set is NOT a measure of memory use. The Commit Charge is. That's why Task Manager Graph from W8 now shows Committed instead of WS.

WS is the amount of - RAM - which the OS Balance Manager has locked for a process or processes. It is NOT load.

32bit applications are allowed to LOCK a maximum 4GB in RAM. 64bit can LOCK a maximum of 8GB.

On x64 (Intel64/AMD64) the WS for 32bit apps is irrelevant because address space available to any process is 256TB. It's only limited by the CPU and available storage.

The memory in use by a 32bit application can easily be more than it's WS, It just doesn't get counted because the 32bit API is 20 years old and pages outside 4GB are deemed to be on a pagefile.

These days half the machines out there have no pagefile and 8GB-16-32GB of RAM.

The system commit charge reported by the CPU however will still climb.

Open RamMap and look in Physical Pages for standby pages, on 32bit apps that's where the missing working set data is.

Regarding RAMMap and VMM.

RAMMapp is fantastic because it shows physical memory addresses (well to an extent). You can literally see where pages are.

Committed Memory in Task Manager is more reliable because that's what the CPU reports. VMM shows pagefile backed for anything outside the process view because a process sees nothing outside it's little virtual world.

Even though it's improved with W7/W8/W10. Task Manager is still unreliable...Resource Monitor in W7 isn't too bad..limited though.

As someone mentioned already since the app doesn't exhaust all resources it's probably not a leak. A leak generally occurs when committed pages are not decommited.

And I just noticed how old the OP question is...

@Rick Brant

What Windows shows as "committed" storage is not RAM. It is a guarantee of availability of storage, should it be needed. If it's ever used the storage may be partly in RAM and partly in the pagefile (assuming you have a pagefile).

Wrong. Committed Memory in Task Manager is reported by the CPU. The only pages it knows about are pages written to physical storage, either RAM or the HDD, because the CPU put them there.

The committed amount in Task Manager is absolutely RAM.

Btw, that screenshot you showed me the other day looks like it was taken on a VM. Which Intel processor do you have that has 3 cores?

Also related: that 2nd test is leak test, the pages were not decommited it doesn't mean they aren't in RAM. PS..What makes you think DB2 is relevant?