How to resolve BROKEN stacks in PerfView for 64 bit process on Windows Server 2012 R2

Question

I'm running perfview (just default collection) on a production system where the CPU is spiking to 100% for a couple of minutes strait. I get some useful results however i also get a bunch of BROKEN stacks.

The machine is Windows Server 2012 R2. The application is an exe that get's launched. The application is compiled in .NET 4.0 however the server is running .NET 4.6.1

The perfview help says that these BROKEN stacks for my situation are fixed in Windows 8 (i'm guessing also Windows Server 2012). My dlls are not ngen-ed but i don't think i have to since, well, the problem is fixed in Windows 8

Any thoughts?

open the ETL in WPA, laod symbols and extend the stacks here.https://channel9.msdn.com/Shows/Defrag-Tools/Defrag-Tools-42-WPT-CPU-Analysis are the stacks also broken? — magicandre1981, Mar 22 '16 at 05:13
I have tried WPA. It does SEEM to have the data but it is significantly harder to use than perfview. I haven't yet been able to use it to find my answers. — Mark, Oct 07 '16 at 19:56

score 4 · Answer 1 · answered Sep 09 '16 at 21:10

4

From PerfView Command Line Help:

Usage: PerfView EnableKernelStacks

On X64 machines if you have problems with broken stacks when the code is executing in the kernel, setting this option and rebooting may improve things

answered Sep 09 '16 at 21:10

Pavel Tsybulivskyi

314
1
10

score 0 · Answer 2 · answered May 13 '22 at 20:45

According to the perfview documentation:

'BROKEN' Stack Frame in Trace- During data collection when an event fires, a stack trace is taken. When this trace is incomplete it is called a broken stack, and can happen for a variety of reasons.

And we can learn also the following reasons why we have some Brokens stacks:

Broken stacks occur for the following reasons 1.In 32 bit processes, ETW relies on the compiler to mark the stack by emitting an 'EBP Frame'. When it fails to do this completely and uses the EBPregister for other purposes, it breaks the stack. This should not happen for operating system code or for .NET Runtime code, but may occur for 3rd party code. 2.In a 32 bit process on a 64 bit Windows 7 or Windows Server 2008 there is a bugin which stacks are uniformly dropped in some sessions. The good news is thatit only happens intermittently. Thus if you collect the data again,it is likely to sidestep this bug. This should be fixed in Windows 8. 3. In a 64-bit process, ETW relies on a different mechanism to walk the stack. In this mechanism, the compiler generates 'unwind information'. Currently this ETW mechanism does not work properly for dynamically generated code(as generated by the .NET runtime JIT compiler). This causes stacks to be broken at the first JIT compiled method on the stack (you see the JIT compile method,but no callers of that method). This issue is fixed on Window8 but not in previous OS versions.
4. Asynchronous activities. Stack crawling is a 'best effort' service. If the sample is taken at a time where it would be impossible to do logging safely,then the OS simply skips it. For example, if during stack crawling whilein the kernel the stack page is found to be swapped out to the disk, then stackcrawling is simply aborted.

How to resolve BROKEN stacks in PerfView for 64 bit process on Windows Server 2012 R2

2 Answers2