4

We have a C# Windows service (runs on Windows 2008 server + .NET framework 4.6 + GC perf hot fix). After running for a few days, the size of bytes in all heaps reaches more than 100 GB (committed), and the private bytes is very high too (110 GB+), but the RAM usage is only 68 GB (Working Set) & 59 GB (Private Bytes). Only 10 GB in page files on this server.

I made a dump and run WinDbg + SOS to analyze memory usage, and I found out that there are a lot of Free objects (about 54 GB). Could this be caused by Free objects? Do those free objects only take up virtual memory but no physical memory? If not, how it is possible that the committed virtual memory is much larger than used physical memory + page files?

Thomas Weller
  • 55,411
  • 20
  • 125
  • 222
Jason
  • 41
  • 3

1 Answers1

2

You have just discovered the concept of demand-zero pages.

Let me cite from Windows Internals, 6th edition, part 2 [Amazon Germany], chapter 10, which is about memory management (page 276 in my edition of the book):

For many of those items, the commit charge may represent the potential use of storage rather than the actual. For example, a page of private committed memory does not actually occupy either a physical page of RAM or the equivalent page file space until it's been referenced at least once. Until then, it's a demand-zero page [...] But commit charge accounts for such pages when the virtual space is first created. This ensures that when the page is later referenced, actual physical storage space will be available for it.

This means: Windows will increase the size of either the working set or the page file when the (committed but yet unaccessed/unused) memory is accessed.

Your sentence

Do those free objects only take up virtual memory but no physical memory?

does not really fit to the rest of the question. Any memory, regardless of what it is filled with (.NET Free objects, .NET regular objects or even C++), may consume physical memory (then it's in the Working Set) or not (then it's in the Page File).

Thomas Weller
  • 55,411
  • 20
  • 125
  • 222
  • Hi Thomas, Thanks for your reply. Demand-zero pages is definitely new concept for me. Have a new direction now. Confusion still continues though. When I sum up all the major .net objects listed by running '!dumpheap -stat', is about 89G(includes 54G Free objects). I think those objects are referenced or used to be referenced, should there be physical memory already allocated for those? – Jason May 25 '16 at 11:28
  • 1
    @Jason: not necessarily. E.g. .NET may "think" your program needs much RAM next, so it pre-allocates say 40 MB. At that time, it's allocated as demand-zero. But then, your program needs 50 MB in a single block (e.g. a byte[]) .NET needs to allocate 50 MB of new memory which gets physical. But the old pre-allocated 40 MB still exist as demand-zero. You could try to implement a sample program to demonstrate such behavior or look whether it can be observed with my [LOH fragmentation sample code](http://stackoverflow.com/a/30361185/480982). Try putting a (native) breakpoint on VitualAlloc(). – Thomas Weller May 25 '16 at 18:25