2

Note: I do not think this question is a duplicate of the "why are there so many ReaderWriterLock objects in my finalization queue?", that question looks to be answered by "It is because the finalization thread is blocked" This question is asking how to tell why the finalization thread is blocked.

I have a .NET windows service that has a memory leak that I believe is caused by the finalize thread being blocked.

Background:

Overview: The service normally runs nonstop between 100 and 300 MB depending on how much data it has queued up. It generally takes 3 + days before I noticed the service's memory is getting out of hand, and will have a Out Of Memory exception within 2 weeks of running.

I think the issue might be related to a massive number of objects that are not leaving the finalization queue.

I have been using WinDbg to try and locate the root of the issue.

Using the !FinalizationQueue command I get the following:

0:002> !FinalizeQueue
SyncBlocks to be cleaned up: 0
Free-Threaded Interfaces to be released: 0
MTA Interfaces to be released: 0
STA Interfaces to be released: 0
----------------------------------
generation 0 has 0 finalizable objects (5b4957bc->5b4957bc)
generation 1 has 0 finalizable objects (5b4957bc->5b4957bc)
generation 2 has 26087 finalizable objects (5b47c020->5b4957bc)
Ready for finalization 17130228 objects (5b4957bc->5f5ee38c)
73ca5b34     1046        20920 System.Threading.ThreadPoolWorkQueueThreadLocals
73cbe424     1065        55380 System.Threading.Thread
73cb6c74     3306        66120 Microsoft.Win32.SafeHandles.SafeRegistryHandle
6627a328     1162       348600 System.Data.DataTable
6627a900    22986      2942208 System.Data.DataColumn
65a376dc  1370770     27415400 System.Transactions.SafeIUnknown
73cb9b88  1421244     28424880 Microsoft.Win32.SafeHandles.SafeAccessTokenHandle
73cc0008 14333858    630689752 System.Threading.ReaderWriterLock

Using Windbg I get the following results looking at the Freachable Queue:

0:002> !sosex.frq -stat
Freachable Queue:
       Count      Total Size   Type
---------------------------------------------------------
     ...
       1              24   System.Security.SafeBSTRHandle
       1              20   System.Security.Cryptography.SafeKeyHandle
       1              16   System.Threading.Gen2GcCallback
       1              20   System.Security.Cryptography.SafeProvHandle
    1038           20760   System.Threading.ThreadPoolWorkQueueThreadLocals
    3299           65980   Microsoft.Win32.SafeHandles.SafeRegistryHandle
 1421227        28424540   Microsoft.Win32.SafeHandles.SafeAccessTokenHandle
      17             340   Microsoft.Win32.SafeHandles.SafeWaitHandle
    1045           54340   System.Threading.Thread
14332692       630638448   System.Threading.ReaderWriterLock
17,130,228 objects, 686,624,360 bytes

I switched to the finalizer thread, and ran the kb command and got:

0:002> kb
 # ChildEBP RetAddr  Args to Child              
00 02eeec70 75f82cc7 000007b0 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
01 02eeece4 75f82c02 000007b0 ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x99
02 02eeecf8 76509839 000007b0 ffffffff 03e86cec KERNELBASE!WaitForSingleObject+0x12
03 (Inline) -------- -------- -------- -------- combase!MTAThreadWaitForCall+0x43 [d:\blue\com\combase\dcomrem\channelb.cxx @ 5657] 
04 02eeed28 765ad524 00712280 00726230 02eeef94 combase!MTAThreadDispatchCrossApartmentCall+0x1ed [d:\blue\com\combase\dcomrem\chancont.cxx @ 193] 
05 (Inline) -------- -------- -------- -------- combase!CRpcChannelBuffer::SwitchAptAndDispatchCall+0x33a2 [d:\blue\com\combase\dcomrem\channelb.cxx @ 5052] 
06 02eeee7c 7648caea 03e86cec 02eeef94 02eeef5c combase!CRpcChannelBuffer::SendReceive2+0x62d [d:\blue\com\combase\dcomrem\channelb.cxx @ 4796] 
07 (Inline) -------- -------- -------- -------- combase!ClientCallRetryContext::SendReceiveWithRetry+0x2e [d:\blue\com\combase\dcomrem\callctrl.cxx @ 1090] 
08 (Inline) -------- -------- -------- -------- combase!CAptRpcChnl::SendReceiveInRetryContext+0x81 [d:\blue\com\combase\dcomrem\callctrl.cxx @ 715] 
09 02eeeedc 764d1789 03e86cec 02eeef94 02eeef5c combase!DefaultSendReceive+0x9e [d:\blue\com\combase\dcomrem\callctrl.cxx @ 671] 
0a (Inline) -------- -------- -------- -------- combase!CAptRpcChnl::SendReceive+0x38 [d:\blue\com\combase\dcomrem\callctrl.cxx @ 752] 
0b 02eeef40 765aa010 03e86cec 02eeef94 02eeef5c combase!CCtxComChnl::SendReceive+0x248 [d:\blue\com\combase\dcomrem\ctxchnl.cxx @ 735] 
0c 02eeef64 75b15769 03ed81c4 02eeefc0 75b15740 combase!NdrExtpProxySendReceive+0x5c [d:\blue\com\combase\ndr\ndrole\proxy.cxx @ 2017] 
0d 02eeef7c 75b96c1b b6aeb22c 03ed81c4 02eef3e0 RPCRT4!NdrpProxySendReceive+0x29
0e 02eef3a8 765a9e1e 7647d8f8 76482ff0 02eef3e0 RPCRT4!NdrClientCall2+0x22b
0f 02eef3c8 7648c46f 02eef3e0 00000008 02eef468 combase!ObjectStublessClient+0x6c [d:\blue\com\combase\ndr\ndrole\i386\stblsclt.cxx @ 215] 
10 02eef3d8 764d1c07 03ed81c4 02eef418 00704070 combase!ObjectStubless+0xf [d:\blue\com\combase\ndr\ndrole\i386\stubless.asm @ 171] 
11 02eef468 764d3024 00704060 74b78aa9 02eef52c combase!CObjectContext::InternalContextCallback+0x1e4 [d:\blue\com\combase\dcomrem\context.cxx @ 4428] 
12 02eef4bc 74b786b1 00704070 74b78aa9 02eef52c combase!CObjectContext::ContextCallback+0xbc [d:\blue\com\combase\dcomrem\context.cxx @ 4332] 
13 02eef5b4 74b78782 74b781d7 02eef644 b618888a clr!CtxEntry::EnterContext+0x243
14 02eef5ec 74b787c3 74b781d7 02eef644 00000000 clr!RCW::EnterContext+0x3a
15 02eef608 74b785a0 00704118 b6188b7e 751063e8 clr!RCWCleanupList::ReleaseRCWListInCorrectCtx+0xc0
16 02eef664 74b419c5 b6188bd2 7510562c 006f0b28 clr!RCWCleanupList::CleanupAllWrappers+0x14d
17 02eef6b4 74b41bc8 006ddfd8 00000001 b6188bee clr!SyncBlockCache::CleanupSyncBlocks+0xd0
18 02eef6cc 74b43ad8 b6188bbe 02eef7e8 02eef860 clr!Thread::DoExtraWorkForFinalizer+0x7f
19 02eef714 74b79b0e 02eef860 006f0b28 02eef860 clr!WKS::GCHeap::FinalizerThreadWorker+0x232
1a 02eef728 74b79b78 b6188aaa 02eef860 00000000 clr!ManagedThreadBase_DispatchInner+0x67
1b 02eef7cc 74b79c45 b618854e 00000000 00000040 clr!ManagedThreadBase_DispatchMiddle+0x7e
1c 02eef828 74af52e2 b61885c2 74af517c 00000000 clr!ManagedThreadBase_DispatchOuter+0x5b
1d 02eef8a4 74b45f91 00000000 00000000 00000000 clr!WKS::GCHeap::FinalizerThreadStart+0x1a0
1e 02eef944 75e57c04 006e5238 75e57be0 b6e30a57 clr!Thread::intermediateThreadProc+0x4d
1f 02eef958 77b4ab8f 006e5238 b4a84c31 00000000 KERNEL32!BaseThreadInitThunk+0x24
20 02eef9a0 77b4ab5a ffffffff 77b30004 00000000 ntdll!__RtlUserThreadStart+0x2f
21 02eef9b0 00000000 74b45f48 006e5238 00000000 ntdll!_RtlUserThreadStart+0x1b

I then switch to my STA thread and ran kb and get:

    0:000> kb
 # ChildEBP RetAddr  Args to Child              
00 0035eb30 75f82cc7 000004cc 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
01 0035eba4 75f82c02 000004cc ffffffff 00000000 KERNELBASE!WaitForSingleObjectEx+0x99
02 0035ebb8 756cfc3a 000004cc ffffffff b4a36eec KERNELBASE!WaitForSingleObject+0x12
03 0035ec78 756cf974 00000000 00000000 00861e98 sechost!ScSendResponseReceiveControls+0xda
04 0035ed28 756d01e4 b4a36ff0 00783af0 006e3de0 sechost!ScDispatcherLoop+0xd1
*** WARNING: Unable to verify checksum for F:\Windows\assembly\NativeImages_v4.0.30319_32\System.Serv759bfb78#\c3cf268acbbaf525b814b58379037630\System.ServiceProcess.ni.dll
05 0035ed64 72de524e 00783af0 b484b419 74ababa0 sechost!StartServiceCtrlDispatcherW+0x58
06 0035ed9c 72de715e 0035ed48 00df322c 00d8c054 System_ServiceProcess_ni+0x2524e
07 0035edd8 72de72ad 00d52734 00d52728 0035edf8 System_ServiceProcess_ni+0x2715e
08 0035ede8 00b99242 00d52734 00d52728 0035ee08 System_ServiceProcess_ni+0x272ad
WARNING: Frame IP not in any known module. Following frames may be wrong.
09 0035edf8 00b9047f 00000000 0035ee20 0035ee14 0xb99242
0a 0035ee08 74ab1396 006e3de0 0035ee68 74ab291f 0xb9047f
0b 0035ee14 74ab291f 0035eeac 0035ee58 74b50281 clr!CallDescrWorkerInternal+0x34
0c 0035ee68 74ab9aff 0035eec0 00d52f84 00000000 clr!CallDescrWorkerWithHandler+0x6b
0d 0035eed8 74b353bf 0035efd4 b4c3929a 0049a880 clr!MethodDescCallSite::CallTargetWorker+0x158
0e 0035effc 74af3e4f 00d52728 00000000 b4c38d76 clr!RunMain+0x193
0f 0035f270 74af4477 00000000 b4c38fe6 001a0000 clr!Assembly::ExecuteMainMethod+0x124
10 0035f778 74af451d b4c38ab6 00000000 00000000 clr!SystemDomain::ExecuteMainMethod+0x651
11 0035f7d0 74af3d1e b4c38576 00000000 00000000 clr!ExecuteEXE+0x4c
12 0035f810 74b0e36d b4c3852a 00000000 00000000 clr!_CorExeMainInternal+0xdc
13 0035f84c 7519cf3f b4d3fc3a 75e57b50 75190000 clr!_CorExeMain+0x4d
14 0035f88c 7521bbcc 7521bb40 7521bb40 0035f8b0 mscoreei!_CorExeMain+0x10a
15 0035f89c 75e57c04 7e97c000 75e57be0 b4380bbf MSCOREE!_CorExeMain_Exported+0x8c
16 0035f8b0 77b4ab8f 7e97c000 b6734d69 00000000 KERNEL32!BaseThreadInitThunk+0x24
17 0035f8f8 77b4ab5a ffffffff 77b30004 00000000 ntdll!__RtlUserThreadStart+0x2f
18 0035f908 00000000 7521bb40 7e97c000 00000000 ntdll!_RtlUserThreadStart+0x1b

Question: How can I get more information about what my STA thread could be doing that could be blocking my finalization thread?

JBdev
  • 353
  • 1
  • 10
  • 1
    I do not think this question is a duplicate of the "why are there so many ReaderWriterLock objects in my finalization queue?", that question looks to be answered by "It is because the finalization thread is blocked" This question is asking how to tell why the finalization thread is blocked. Thanks! – JBdev May 08 '17 at 21:37
  • 5
    "my STA thread" is the problem. It is not an STA thread. When you use the [STAThread] attribute on your Main() method then you make a *promise*, cross your heart, hope to die. You did not keep the promise so you died. An STA thread must pump a message loop and must never block. You don't have a message loop and you block. You have to delete the attribute. If the COM server you use gets upset about it then you have to [provide a hospitable home for it](http://stackoverflow.com/a/21684059/17034). – Hans Passant May 09 '17 at 08:14
  • Hans, Thanks for your response, I have been reading some of your other answers and they have been helpful. I am trying to find good information about how to correctly change my code. I see a couple options, I can change my STA thread to MTA, and simply start a temporary STA thread for GUI configuration, or I can correctly implement my STA thread with pumping and a message loop. Question can you point me to information on how to correctly implement a STA thread in C#? My searches are not returning much. – JBdev May 09 '17 at 15:43
  • I already did. No real idea what else you could need, you did not tell us anything about your code. "GUI configuration" is fairly baffling, a service cannot have a GUI. – Hans Passant May 09 '17 at 15:48

0 Answers0