!heap -stat -h doesn't show allocations

Question

I'm trying to determine why my applications consumes 4GB of Private Bytes. So I took a full memory dump, loaded it in windbg. But analyzing using !heap -stat -h produces weird results which don't add up:

0:000> !heap -s
(...)
          Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                            (k)     (k)    (k)     (k) length      blocks cont. heap 
-------------------------------------------------------------------------------------
000002d0a0000000 00000002 2800804 2780508 2800700   2984  1980   177    0      6   LFH
000002d09fe10000 00008000      64      4     64      2     1     1    0      0      
000002d09ff70000 00001002 1342924 1334876 1342820  13042  3342    87    0      0   LFH

Ok, I got a 2.8GB heap and a 1.34GB heap. Let's look at the allocations of the first one:

0:000> !heap -stat -h 000002d0a0000000 
 heap @ 000002d0a0000000
group-by: TOTSIZE max-display: 20
    size     #blocks     total     ( %) (percent of total busy bytes)
    651 291 - 1035e1  (16.00)
    79c 1df - e3ce4  (14.06)
    28 156d - 35908  (3.31)
    (...)

IIUC, the first line means block size 0x651(=1617 bytes), number blocks 0x291(=657), for total bytes of 0x103531(=1062369 bytes =~1MB), and that's 16% of total busy bytes. But looking at the summary, there should be ~2.8GB of busy bytes!

Another disparity:

0:000> !heap -stat -h 000002d0a0000000 -grp A
 heap @ 000002d0a0000000
group-by: ALLOCATIONSIZE max-display: 20
    size     #blocks    total     ( %) (percent of total busy bytes)
    a160 1 - a160  (0.62)
    7e50 2 - fca0  (0.97)

0:000> !heap -h 000002d0a0000000
(...)
(509 lines that note allocations with size 7e50, like this one:)
    000002d0a3f48000: 11560 . 07e60 [101] - busy (7e50)

Edit: Many lines also say Internal at the end, which appears to mean HEAP_ENTRY_VIRTUAL_ALLOC - but the 509 lines with (7e50) don't.

My question: How can I get !heap -stat -h to show all the allocations, so they add up to the output of !heap -s?

comments from a fleeting glance stat -h takes a handle doesn't it ? not a heap address like you use iirc — blabb, Nov 16 '15 at 16:42
@blabb: According the documentation, you're right. But it seems the handle correlates to the heap address. I wrote a short test in C++ and `HeapCreate()` returned the same handle value as displayed by `!heap -s`. — Thomas Weller, Nov 16 '15 at 21:42

score 4 · Answer 1 · answered Nov 16 '15 at 23:41

At the moment I can only explain the busy percentage, but that may already be helpful. Its value is a bit misleading.

Virtual memory is memory taken from VirtualAlloc(). The C++ heap manager uses that basic mechanism to get memory from the operating system. That virtual memory can be committed (ready to use) or reserved (can be committed later). The output of !heap -s tells you the status of the heaps with respect to that virtual memory.

So we agree that any memory the C++ heap manager can use is committed memory. This coarse granular virtual memory is split into finer blocks by the C++ heap manager. The heap manager may allocate such smaller blocks and free them, depending on the need of malloc()/free() or new/delete operations.

When blocks become free, they are no longer busy. At the same time, the C++ heap manager may decide to not give the free block back to the OS, because

it can't, since other parts of the 64k virtual memory are still in use
or it doesn't want to (internal reasons we can't exactly know, e.g. performance reasons)

Since the free parts do not count as busy, the busy percentage seems to be too high when compared to the virtual memory.

Mapped to your case, this means:

you have 2.8 GB of virtual memory
in heap 000002d0a0000000, you have ~1 MB / 16% = 6.25 MB of memory in use, the rest could be in free heap blocks (it possibly isn't)

The following example is based on this C++ code:

#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <string>
#include <iomanip>

int main()
{
    HANDLE hHeap = HeapCreate(0, 0x1000000, 0x10000000); // no options, initial 16M, max 256M
    HeapAlloc(hHeap, HEAP_GENERATE_EXCEPTIONS, 511000); // max. allocation size for non-growing heap
    std::cout << "Debug now, handle is 0x" << std::hex << std::setfill('0') << std::setw(sizeof(HANDLE)) << hHeap << std::endl;
    std::string dummy;
    std::getline(std::cin, dummy);
    return 0;
}

The only 511kB block will be reported as 100%, although it is only ~1/32 of the 16 MB:

0:001> !heap -stat -h 009c0000
 heap @ 009c0000
group-by: TOTSIZE max-display: 20
    size     #blocks     total     ( %) (percent of total busy bytes)
    7cc18 1 - 7cc18  (100.00)

To see the free parts as well, use !heap -h <heap> -f:

0:001> !heap -h 0x01430000 -f
Index   Address  Name      Debugging options enabled
  3:   01430000 
    Segment at 01430000 to 11430000 (01000000 bytes committed)
    Flags:                00001000
    ForceFlags:           00000000
    Granularity:          8 bytes
    Segment Reserve:      00100000
    Segment Commit:       00002000
    DeCommit Block Thres: 00000200
    DeCommit Total Thres: 00002000
    Total Free Size:      001f05c7
    Max. Allocation Size: 7ffdefff
    Lock Variable at:     01430138
    Next TagIndex:        0000
    Maximum TagIndex:     0000
    Tag Entries:          00000000
    PsuedoTag Entries:    00000000
    Virtual Alloc List:   014300a0
    Uncommitted ranges:   01430090
    FreeList[ 00 ] at 014300c4: 01430590 . 0240e1b0  
        0240e1a8: 7cc20 . 21e38 [100] - free             <-- no. 1
        02312588: 7f000 . 7f000 [100] - free             <-- no. 2
        [...]
        01430588: 00588 . 7f000 [100] - free             <-- no. 32

    Heap entries for Segment00 in Heap 01430000
         address: psize . size  flags   state (requested size)
        01430000: 00000 . 00588 [101] - busy (587)
        01430588: 00588 . 7f000 [100]
        [...]
        02312588: 7f000 . 7f000 [100]
        02391588: 7f000 . 7cc20 [101] - busy (7cc18)
        0240e1a8: 7cc20 . 21e38 [100]
        0242ffe0: 21e38 . 00020 [111] - busy (1d)
        02430000:      0f000000      - uncommitted bytes.
0:001> ? 7cc18
Evaluate expression: 511000 = 0007cc18

Here we see that I have a heap of 256 MB (240 MB uncommitted, 0x0f000000 + 16 MB committed, 0x01000000). Summing up the items in the FreeList, I get

0:001> ? 0n31 * 7f000 + 21e38 
Evaluate expression: 16264760 = 00f82e38

So almost everything (~16 MB) is considered as free and not busy by the C++ heap manager. Memory like that 16 MB is reported by !heap -s in this way in WinDbg 6.2.9200:

0:001> !heap -s
LFH Key                   : 0x23e41d0e
Termination on corruption : ENABLED
  Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                    (k)     (k)    (k)     (k) length      blocks cont. heap 
-----------------------------------------------------------------------------
004d0000 00000002    1024    212   1024      6     5     1    0      0   LFH
00750000 00001002      64     20     64      9     2     1    0      0      
01430000 00001000  262144  16384 262144  15883    32     1    0      0      
    External fragmentation  96 % (32 free blocks)
-----------------------------------------------------------------------------

IMHO there's a bug regarding reserved and committed memory: it should be 262144k virtual - 16384 committed = 245760k reserved.

Note how the list length matches the number of free blocks reported before.

Above explains the busy percentage only. The remaining question is: the free memory reported in your case doesn't match this scenario.

Usually I'd say the remaining memory is in virtual blocks, i.e. memory blocks that are larger than 512 kB (32 bit) or 1 MB (64 bit) as mentioned on MSDN for growable heaps. But that's not the case here.

There is no output about virtual blocks and the number of virtual blocks is reported as 0.

A program that generates a virtual block would be

#include "stdafx.h"
#include <iostream>
#include <Windows.h>
#include <string>
#include <iomanip>

int main()
{
    HANDLE hHeap = HeapCreate(0, 0x1000000, 0); // no options, initial 16M, growable
    HeapAlloc(hHeap, HEAP_GENERATE_EXCEPTIONS, 20*1024*1024); // 20 MB, force growing
    std::cout << "Debug now, handle is 0x" << std::hex << std::setfill('0') << std::setw(sizeof(HANDLE)) << hHeap << std::endl;
    std::string dummy;
    std::getline(std::cin, dummy);
    return 0;
}

and the !heap command would mention the virtual block:

0:001> !heap -s
LFH Key                   : 0x7140028b
Termination on corruption : ENABLED
  Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                    (k)     (k)    (k)     (k) length      blocks cont. heap 
-----------------------------------------------------------------------------
006d0000 00000002    1024    212   1024      6     5     1    0      0   LFH
001d0000 00001002      64     20     64      9     2     1    0      0      
Virtual block: 01810000 - 01810000 (size 00000000)
00810000 00001002   16384  16384  16384  16382    33     1    1      0      
    External fragmentation  99 % (33 free blocks)
-----------------------------------------------------------------------------

In your case however, the value virtual blocks is 0. Perhaps this what is reported as "Internal" in your version of WinDbg. If you have not upgraded yet, try version 6.2.9200 to get the same output as I do.

WinDbg version is later than 6.2.9200 (I have access to insider builds). I will test your virtual block program and see whether I get the same results. — Jonathan, Nov 17 '15 at 10:34
I've tested the virtual block programs. I did get the "External fragmentation" note, but not the "Virtual block" - probably because my app and test programs are 64-bit, where the thresholds are different. Anyways, I ended up finding the memory issue using [UMDH](https://msdn.microsoft.com/en-us/library/windows/hardware/ff558947(v=vs.85).aspx) — Jonathan, Nov 17 '15 at 13:47

!heap -stat -h doesn't show allocations

1 Answers1