Why does User-Mode Dump Heap (UMDH) not detect BSTR-Leaks in x64 applications?

Question

I've had great success in finding memory leaks via GFLAGS and UMDH. However, I just discovered that UMDH somehow does not see BSTR-Leaks in 64 bit applications. Does anyone know why that is? Is it a bug in UMDH? A change in how the Heap-Manager works for x64 applications? Where can I even report such issues to Microsoft?

I investigated this issue with a small test executable like this:

int main()
{
    while (true)
    {
        BSTR sLeak = CComBSTR("I am a leaky BSTR. Please find me!").Detach();
        Sleep(16);
    }

    return 0;
}

Here is the UMDH-output for the x86 test executable. The leak is visible!

+ 2878464 ( 3597312 - 718848)   2342 allocs BackTrace53973BC
+    1874 (   2342 -    468)    BackTrace53973BC    allocations

    ntdll!RtlWalkHeap+194
    ntdll!RtlAllocateHeap+10DC
    ntdll!RtlAllocateHeap+3E
    combase!PropVariantCopy+456
    OLEAUT32!SysAllocString+10F
    OLEAUT32!SysAllocStringLen+40
    LeakTestCom!main+48 (d:\...\leaktestcom\leaktestcom.cpp, 78)
    LeakTestCom!__scrt_common_main_seh+FA (d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl, 288)
    KERNEL32!BaseThreadInitThunk+19
    ntdll!RtlGetAppContainerNamedObjectPath+11E
    ntdll!RtlGetAppContainerNamedObjectPath+EE

-     288 (      0 -    288)      0 allocs  BackTrace26DADAC
-       3 (      0 -      3)    BackTrace26DADAC    allocations

    ntdll!RtlWalkHeap+194
    ntdll!RtlAllocateHeap+10DC
    ntdll!RtlAllocateHeap+3E
    ntdll!TpSetWaitEx+552
    ntdll!TpCallbackIndependent+208
    KERNEL32!BaseThreadInitThunk+19
    ntdll!RtlGetAppContainerNamedObjectPath+11E
    ntdll!RtlGetAppContainerNamedObjectPath+EE

-    1380 (      0 -   1380)      0 allocs  BackTrace26DAE00
-       3 (      0 -      3)    BackTrace26DAE00    allocations

    ntdll!RtlWalkHeap+194
    ntdll!RtlAllocateHeap+10DC
    ntdll!RtlAllocateHeap+3E
    ntdll!TpCallbackIndependent+7F0
    KERNEL32!BaseThreadInitThunk+19
    ntdll!RtlGetAppContainerNamedObjectPath+11E
    ntdll!RtlGetAppContainerNamedObjectPath+EE


Total increase == 2876796 requested + 134772 overhead = 3011568

And here is the UMDH output for the x64 test executable. As you can see, the BSTR allocations are not visible.

-     136 (      0 -    136)      0 allocs  BackTraceD670D5C4
-       1 (      0 -      1)    BackTraceD670D5C4   allocations

    ntdll!RtlAllocateHeap+AFD
    ntdll!TpReleaseCleanupGroupMembers+1D51
    ntdll!TpReleaseCleanupGroupMembers+4F0
    KERNEL32!BaseThreadInitThunk+14
    ntdll!RtlUserThreadStart+21

-    2736 (      0 -   2736)      0 allocs  BackTraceD670DA44
-       3 (      0 -      3)    BackTraceD670DA44   allocations

    ntdll!RtlAllocateHeap+AFD
    ntdll!TpReleaseCleanupGroupMembers+B5C
    KERNEL32!BaseThreadInitThunk+14
    ntdll!RtlUserThreadStart+21


Total decrease ==   2872 requested +    200 overhead =   3072

It could be related to the fact that BSTR memory is allocated using ::SysAllocString (COM memory pool) and thus not using the heap. https://learn.microsoft.com/en-us/windows/win32/api/oleauto/nf-oleauto-sysallocstring. Do you have issues with in process client code not calling SysFreeString? ( — Pepijn Kramer, Dec 22 '21 at 14:37
@PepijnKramer Yeah, that's what I thought as well, but why would it work for x86 binaries, but not for x64 binaries then? The reason for my question is that we recently had a BSTR-Leak in our COM-based software and I need proper tools to find such issues. I tested numerous memory debugging tools and most are way too slow or don't show the COM-leaks either. UMDH on the contrary is really leightweight (almost zero performance impact) and easy to use. — STiFU, Dec 22 '21 at 17:09
No idea why x86 and x64 would behave differently and I don't know UMDH. So other then hunting for "naked" BSTR's (and ::SysAllocString and ::SysFreeString calls) in source code (and replacing them with ATL::CComBSTR) I wouldn't have any useful advice at the moment. They only place where you should then find CComBSTR::Detach is in the COM call implementations themselves where the string is handed over to an BSTR* out param. — Pepijn Kramer, Dec 22 '21 at 18:24
Oh looking a bit more at the calls in x64.... I think the x64 compiler version actually has optimized away the whole BSTR allocation since you are not using the string at all. Maybe you could try to output the string to std::wcout see if that makes a difference since then you would be using the allocated string. — Pepijn Kramer, Dec 22 '21 at 18:27
Nah, that's not it. I tried an x64 debug build as well and the result is the same. Replacing all occurences of BSTR with CComBSTR is sadly not an option. Furthermore, I fear UMDH does not see other COM-Allocation methods either in x64, which is a huge problem for our software. — STiFU, Dec 22 '21 at 18:47

Why does User-Mode Dump Heap (UMDH) not detect BSTR-Leaks in x64 applications?

0 Answers0