2

I have a service that is reporting a large amount of logical threads. From PerfMon:

.NET CLR LocksAndThreads -> # of current logical threads: 663
.NET CLR LocksAndThreads -> # of current physical threads: 659
Process -> Thread Count: 15

This is too high, so I captured a memory dump (via sysinternals procdump.exe) and opened it from Visual Studio (Debug with Mixed). Once everything is loaded up, I looked in the threads window, and it only shows the 15 OS threads, not the .net physical or .net logical. The service itself is a windows service that hosts 4 WCF services (System.ServiceModel.ServiceHost).

How do I find out what these threads are, so that I can fix the code and get rid of them? How do I get the logical threads to be recognized and displayed by visual studio? Is it a problem with Visual Studio, or a problem with the dump itself?

Justin Killen
  • 728
  • 6
  • 19
  • If you use the standard CLR host, then you have 1 to 1 mapping between logical and physical threads. To get more assistance you must provide more information about your service host. – Hamlet Hakobyan Jun 19 '14 at 18:51
  • This isn't possible. Wrong timing on exactly when you made the minidump perhaps. – Hans Passant Jun 19 '14 at 19:43
  • @HansPassant what exactly isn't possible? I'm reading the values straight from perfmon from a live feed. The values are consistently rising (although slowly), and haven't been less than 400 for over 16 hours now. I took another memory dump just a few minutes ago, and the threads listed in visual studio matched the count from Processes -> Thread Count. – Justin Killen Jun 19 '14 at 20:35
  • I've managed to get the dump loaded in WinDbg and list out the threads via the !threads command. It lists ThreadCount: 696 and DeadThread: 687. The exception column for most of the threads is listed as "Threadpool Completion Port". I'm not sure what that means - still researching – Justin Killen Jun 19 '14 at 21:51
  • I found a leak that was holding a reference to the WCF thread. The thread stopped, but couldn't be GC'd because of the reference. – Justin Killen Jun 20 '14 at 15:39
  • Useful discussion: http://stackoverflow.com/questions/17945845/how-to-diagnose-what-is-keeping-dead-threads-from-being-recycled-in-net – Justin Killen Jun 20 '14 at 15:40
  • 1
    Can you answer the own question than, please? I think It would be great for the community to have a short description about how to fix/investigate the issue. – Alex Netkachov Jun 22 '14 at 00:05

1 Answers1

3

First, you need to obtain a memory dump. There are a variety of methods to do this. The easiest one that I've found is procdump.exe, part of SysInternals, available with documentation here.

Next, you need to download and install WinDbg and get SOS working (SOS is the module that let's you view .net managed processes). More information on setting that up is available here. If you got your dump from a server (as was my case), then the .net versions might be slightly off, and SOS won't be able to work correctly with the dump file. If that happens, you'll need to copy a few relevant files from the source .net version to your windbg installation as per this comment.

Once everything is setup and you have the dump loaded, run the command: 

!threads

This should give you a list of .net logical threads. Also of note is that before the list of threads, it will give you a summary. Of importance for this question, DeadThread was extremely high (600+). This indicates that the threads have completed, but the memory they are holding (the stack) can't be released.

In my specific case, I found that the threads that were getting hung were threads from the WCF thread pool that couldn't GC because other threads were holding references to it. I changed the other threads to null out the parent thread when they didn't need it anymore, , and that allowed it to get GC'd correctly.

Community
  • 1
  • 1
Justin Killen
  • 728
  • 6
  • 19