2

I wrote a program with Visual C++ and it crashes Windows. Luckily, I have a crash dump!

!analyze -v states that the faulty process is indeed my program and shows me a kernel-mode call stack. However, I need to understand the state of my user-mode program to make sense of it.

According to some other answers here (e.g. this one), it should just work.

But I cannot get any such information out of windbg. All call stacks simply end at nt!KiSystemServiceCopyEnd+0x28. As a minimal example, the call stack for an idle worker thread of mine from the standard Win32 thread pool:

nt!KiSwapContext+0x76
nt!KiSwapThread+0xbfd
nt!KiCommitThreadWait+0x144
nt!KeRemoveQueueEx+0x27e
nt!IoRemoveIoCompletion+0x99
nt!NtWaitForWorkViaWorkerFactory+0x25e
nt!KiSystemServiceCopyEnd+0x28 (TrapFrame @ ffff820d`2a2dfb00)
0x00007ff8`08a1fa54

I would at least expect RtlUserThreadStart somewhere, but nothing in sight.

The program in question was compiled in Visual C++ 2019 for x64 with debug information. The program and the PDB files are still in place, untouched since the crash. The Visual Studio debugger was attached at the time of the crash, but I have another one without a debugger and it’s no better.

Interestingly, if I select other processes which ran during the crash and if I happen to hit one which wasn’t paged out (e.g. firefox), their callstacks end at nt!KiSystemServiceCopyEnd+0x28 as well.

What can I do to retrieve the state of my program during the crash?

Krishty
  • 164
  • 12
  • 2
    was different dump types - full memory, kernel memory only, mini-dump. you not say - what dump type (windbg write this at begin). probably you have kernel-memory dump or mini-dump. in this case user mode memory simply not included to dump and impossible show any user mode stack – RbMm Oct 31 '20 at 21:59
  • Indeed: `Kernel Bitmap Dump File: Kernel address space is available, User address space may not be available.` The output was overwhelming, I think … Do you want to post this as an answer? – Krishty Oct 31 '20 at 22:05
  • in this case - no user space memory in dump and not possible show user-mode stack. but what is bsod ? are your user mode code lead to bsod ?! or crash was in kernel ? what is happens ? or this is critical process terminated crash ? – RbMm Oct 31 '20 at 22:07
  • It’s definitely my user-mode calls. No BSOD, but the systems hangs for a few seconds and dumps kernel core. It does continue normally afterwards, so I assume it hits a non-critical assertion or something. I wanted to understand the program state so I can reproduce it reliably. – Krishty Oct 31 '20 at 22:10
  • You say: "it crashes Windows" - that would be a BSOD. Then you say "No BSOD, but the systems hangs for a few seconds and dumps kernel core." Could you post the code that causes this issue so we can reproduce it? What are you developing? Is it a driver or a keyboard sniffer? – Thomas Weller Nov 02 '20 at 07:08
  • I’m developing a Win32 GUI application. I create a few windows and move them around. Sometimes this makes the system hang for a few seconds (screen+cursor freeze), during which the kernel dumps core. The system does resume normal operation afterwards, but the kernel dump is explicit in that I performed an “impossible” operation which triggered a bugcheck in `Win32k.sys`. The issue is very rare (ca. once every two weeks) but it does show on two very different systems. I want to understand the core dumps to reproduce it reliably and submit a bug report, if necessary. – Krishty Nov 02 '20 at 07:54
  • I've never see that in my 10 year debugging career. Very interesting. – Thomas Weller Nov 03 '20 at 08:28
  • @andI'msureI'mmissingsome in case you’re still interested in the details of the bugcheck, I’ve uploaded all information here: https://zfx.info/viewtopic.php?f=4&t=4705&p=63826#p63826 – Krishty Nov 04 '20 at 21:16
  • Thanks for the update. In the end it's just another crappy graphics driver? – Thomas Weller Nov 05 '20 at 07:26
  • 1
    @ThomasWeller Possible, but a) the machine has no signs of other dumps whatsoever; b) the problem persists through at least 4 graphics driver updates; c) I observed similar behavior on a very different machine, which I’m going to revisit this weekend. – Krishty Nov 05 '20 at 07:47
  • @ThomasWeller the problem vanished after upgrading to Windows 10 20H2. I did install the exact same graphics driver as before (I always back up my drivers), so I assume it was a kernel bug. – Krishty Apr 13 '21 at 19:21

1 Answers1

3

RbMm was right – the dump doesn’t include any user address space, so there’s nothing for windbg to show.

When opening a dump file with windbg, watch the first few lines of output. There should be a message like Kernel Bitmap Dump File: Kernel address space is available, User address space may not be available, which says it all.

Krishty
  • 164
  • 12
  • Perhaps the amount of memory is affected by the "usual" kernel dump settings. If you change it to "full kernel dump", you might get the info you want – Thomas Weller Nov 03 '20 at 08:43
  • @andI'msureI'mmissingsome I hope that as well and changed the setting to *complete*. It’ll take a few weeks to know because the problem is so rare. FWIW, the dump is written by `nt!DbgkWerCaptureLiveKernelDump → nt!DbgkpWerProcessPolicyResult → nt!DbgkpWerCaptureLiveFullDump` – if anyone knows which settings affect that, I’d be glad to learn. – Krishty Nov 03 '20 at 09:45
  • According to [this analysis](https://www.osr.com/nt-insider/2019-issue1/live-kernel-reports/), `DbgkWerCaptureLiveKernelDump` writes at most one crash dump over a period of days. So it is possible that I’m hitting the bugcheck all the time, but only once in two weeks the kernel actually generates a dump file. That’s not helpful in reducing the problem, at all :-( Guess I’ll disclose details then. – Krishty Nov 03 '20 at 09:59
  • See https://learn.microsoft.com/en-us/windows-hardware/drivers/devtest/dtrace-live-dump void DbgWerCaptureLiveKernelDump(LPCWSTR ComponentName, ULONG BugCheckCode, ULONG_PTR PoolTag, const char* FormatString, va_list args, ULONG_PTR unknown, ULONG DumpFlags); – Stephen Eckels Jun 17 '22 at 01:21