9

I'm working on a complex WPF application which hangs in production once several days. There's a thread other than GUI thread filling data to models bind to the grid and triggers INotifyPropertyChanged.PropertyChanged event. I wrote a script to attach MDbg to the hanging process and dumping the current stack trace of the threads. It helps a lot when finding the cause of deadlock but it doesn't help this time.

The thread which is updating models is stopped at acquiring ReadLock:

Thread [#:8]
*0. WindowsBase.dll#0!MS.Internal.ReaderWriterLockWrapper.get_ReadLock()  (source line information unavailable)
 1. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(sender=<...>, args=System.ComponentModel.PropertyChangedEventArgs)  (source line information unavailable)
 2. ( ... firing PropertyChanged event ... )

The same thing happens to the GUI thread:

Thread [#:0]
*0. WindowsBase.dll#0!MS.Internal.ReaderWriterLockWrapper.get_ReadLock()  (source line information unavailable)
 1. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(sender=MyCompany.Windows.ViewModel.Window.WindowViewModel, args=System.ComponentModel.PropertyChangedEventArgs)  (source line information unavailable)
 2. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Model.ViewModelItemBase.NotifyPropertyChanged(propertyName="IsActive")  (source line information unavailable)
 3. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Window.WindowViewModel.set_IsActive(value=True)  (source line information unavailable)
 4. MyCompany.Windows.dll#0!MyCompany.Windows.ViewModel.Window.IsActiveBinding.OnWindowIsActiveChanged(sender=MyCompany.Xpf.Views.XpfRibbonShell.XpfRibbonShellView, e=System.EventArgs)  (source line information unavailable)
 5. WindowsBase.dll#0!MS.Internal.ComponentModel.PropertyChangeTracker.OnPropertyInvalidation(d=<N/A>, args=<N/A>)  (source line information unavailable)
 6. WindowsBase.dll#0!System.Windows.DependentList.InvalidateDependents(source=<N/A>, sourceArgs=<N/A>)  (source line information unavailable)
 7. WindowsBase.dll#0!System.Windows.DependencyObject.NotifyPropertyChange(args=<N/A>)  (source line information unavailable)
 8. WindowsBase.dll#0!System.Windows.DependencyObject.UpdateEffectiveValue(entryIndex=<N/A>, dp=<N/A>, metadata=<N/A>, oldEntry=<N/A>, newEntry=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>)  (source line information unavailable)
 9. WindowsBase.dll#0!System.Windows.DependencyObject.SetValueCommon(dp=<N/A>, value=<N/A>, metadata=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>, isInternal=<N/A>)  (source line information unavailable)
 10. WindowsBase.dll#0!System.Windows.DependencyObject.SetValue(key=<N/A>, value=<N/A>)  (source line information unavailable)
 11. PresentationFramework.dll#0!System.Windows.Window.HandleActivate(windowActivated=<N/A>)  (source line information unavailable)
 12. PresentationFramework.dll#0!System.Windows.Window.WmActivate(wParam=<N/A>)  (source line information unavailable)
 13. PresentationFramework.dll#0!System.Windows.Window.WindowFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 14. PresentationCore.dll#0!System.Windows.Interop.HwndSource.PublicHooksFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 15. WindowsBase.dll#0!MS.Win32.HwndWrapper.WndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 16. WindowsBase.dll#0!MS.Win32.HwndSubclass.DispatcherCallbackOperation(o=<N/A>)  (source line information unavailable)
 17. WindowsBase.dll#0!System.Windows.Threading.ExceptionWrapper.InternalRealCall(callback=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 18. WindowsBase.dll#0!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(source=System.Windows.Threading.Dispatcher, method=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<null>)  (source line information unavailable)
 19. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WrappedInvoke(callback=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<N/A>)  (source line information unavailable)
 20. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.InvokeImpl(priority=<N/A>, timeout=<N/A>, method=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 21. WindowsBase.dll#0!MS.Win32.HwndSubclass.SubclassWndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>)  (source line information unavailable)
    [IL Method without Metadata]
 22. WindowsBase.dll#0!MS.Internal.ReaderWriterLockWrapper.get_ReadLock()  (source line information unavailable)
 23. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(sender=MyCompany.Windows.ViewModel.Window.WindowViewModel, args=System.ComponentModel.PropertyChangedEventArgs)  (source line information unavailable)
 24. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Model.ViewModelItemBase.NotifyPropertyChanged(propertyName="IsActive")  (source line information unavailable)
 25. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Window.WindowViewModel.set_IsActive(value=False)  (source line information unavailable)
 26. MyCompany.Windows.dll#0!MyCompany.Windows.ViewModel.Window.IsActiveBinding.OnWindowIsActiveChanged(sender=MyCompany.Xpf.Views.XpfRibbonShell.XpfRibbonShellView, e=System.EventArgs)  (source line information unavailable)
 27. WindowsBase.dll#0!MS.Internal.ComponentModel.PropertyChangeTracker.OnPropertyInvalidation(d=<N/A>, args=<N/A>)  (source line information unavailable)
 28. WindowsBase.dll#0!System.Windows.DependentList.InvalidateDependents(source=<N/A>, sourceArgs=<N/A>)  (source line information unavailable)
 29. WindowsBase.dll#0!System.Windows.DependencyObject.NotifyPropertyChange(args=<N/A>)  (source line information unavailable)
 30. WindowsBase.dll#0!System.Windows.DependencyObject.UpdateEffectiveValue(entryIndex=<N/A>, dp=<N/A>, metadata=<N/A>, oldEntry=<N/A>, newEntry=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>)  (source line information unavailable)
 31. WindowsBase.dll#0!System.Windows.DependencyObject.SetValueCommon(dp=<N/A>, value=<N/A>, metadata=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>, isInternal=<N/A>)  (source line information unavailable)
 32. WindowsBase.dll#0!System.Windows.DependencyObject.SetValue(key=<N/A>, value=<N/A>)  (source line information unavailable)
 33. PresentationFramework.dll#0!System.Windows.Window.HandleActivate(windowActivated=<N/A>)  (source line information unavailable)
 34. PresentationFramework.dll#0!System.Windows.Window.WmActivate(wParam=<N/A>)  (source line information unavailable)
 35. PresentationFramework.dll#0!System.Windows.Window.WindowFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 36. PresentationCore.dll#0!System.Windows.Interop.HwndSource.PublicHooksFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 37. WindowsBase.dll#0!MS.Win32.HwndWrapper.WndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 38. WindowsBase.dll#0!MS.Win32.HwndSubclass.DispatcherCallbackOperation(o=<N/A>)  (source line information unavailable)
 39. WindowsBase.dll#0!System.Windows.Threading.ExceptionWrapper.InternalRealCall(callback=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 40. WindowsBase.dll#0!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(source=System.Windows.Threading.Dispatcher, method=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<null>)  (source line information unavailable)
 41. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WrappedInvoke(callback=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<N/A>)  (source line information unavailable)
 42. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.InvokeImpl(priority=<N/A>, timeout=<N/A>, method=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 43. WindowsBase.dll#0!MS.Win32.HwndSubclass.SubclassWndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>)  (source line information unavailable)
    [IL Method without Metadata]
 44. WindowsBase.dll#0!MS.Internal.ReaderWriterLockWrapper.get_ReadLock()  (source line information unavailable)
 45. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.OnPropertyChanged(sender=MyCompany.Windows.ViewModel.Window.WindowViewModel, args=System.ComponentModel.PropertyChangedEventArgs)  (source line information unavailable)
 46. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Model.ViewModelItemBase.NotifyPropertyChanged(propertyName="IsActive")  (source line information unavailable)
 47. MyCompany.Windows.Contracts.dll#0!MyCompany.Windows.ViewModel.Window.WindowViewModel.set_IsActive(value=True)  (source line information unavailable)
 48. MyCompany.Windows.dll#0!MyCompany.Windows.ViewModel.Window.IsActiveBinding.OnWindowIsActiveChanged(sender=MyCompany.Xpf.Views.XpfRibbonShell.XpfRibbonShellView, e=System.EventArgs)  (source line information unavailable)
 49. WindowsBase.dll#0!MS.Internal.ComponentModel.PropertyChangeTracker.OnPropertyInvalidation(d=<N/A>, args=<N/A>)  (source line information unavailable)
 50. WindowsBase.dll#0!System.Windows.DependentList.InvalidateDependents(source=<N/A>, sourceArgs=<N/A>)  (source line information unavailable)
 51. WindowsBase.dll#0!System.Windows.DependencyObject.NotifyPropertyChange(args=<N/A>)  (source line information unavailable)
 52. WindowsBase.dll#0!System.Windows.DependencyObject.UpdateEffectiveValue(entryIndex=<N/A>, dp=<N/A>, metadata=<N/A>, oldEntry=<N/A>, newEntry=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>)  (source line information unavailable)
 53. WindowsBase.dll#0!System.Windows.DependencyObject.SetValueCommon(dp=<N/A>, value=<N/A>, metadata=<N/A>, coerceWithDeferredReference=<N/A>, coerceWithCurrentValue=<N/A>, operationType=<N/A>, isInternal=<N/A>)  (source line information unavailable)
 54. WindowsBase.dll#0!System.Windows.DependencyObject.SetValue(key=<N/A>, value=<N/A>)  (source line information unavailable)
 55. PresentationFramework.dll#0!System.Windows.Window.HandleActivate(windowActivated=<N/A>)  (source line information unavailable)
 56. PresentationFramework.dll#0!System.Windows.Window.WmActivate(wParam=<N/A>)  (source line information unavailable)
 57. PresentationFramework.dll#0!System.Windows.Window.WindowFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 58. PresentationCore.dll#0!System.Windows.Interop.HwndSource.PublicHooksFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 59. WindowsBase.dll#0!MS.Win32.HwndWrapper.WndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 60. WindowsBase.dll#0!MS.Win32.HwndSubclass.DispatcherCallbackOperation(o=<N/A>)  (source line information unavailable)
 61. WindowsBase.dll#0!System.Windows.Threading.ExceptionWrapper.InternalRealCall(callback=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 62. WindowsBase.dll#0!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(source=System.Windows.Threading.Dispatcher, method=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<null>)  (source line information unavailable)
 63. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WrappedInvoke(callback=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<N/A>)  (source line information unavailable)
 64. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.InvokeImpl(priority=<N/A>, timeout=<N/A>, method=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 65. WindowsBase.dll#0!MS.Win32.HwndSubclass.SubclassWndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>)  (source line information unavailable)
    [IL Method without Metadata]
 66. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.PrivateAddListener(source=<N/A>, listener=<N/A>, propertyName=<N/A>)  (source line information unavailable)
 67. PresentationFramework.dll#0!MS.Internal.Data.PropertyPathWorker.ReplaceItem(k=<N/A>, newO=<N/A>, parent=<N/A>)  (source line information unavailable)
 68. PresentationFramework.dll#0!MS.Internal.Data.PropertyPathWorker.UpdateSourceValueState(k=<N/A>, collectionView=<N/A>, newValue=<N/A>, isASubPropertyChange=<N/A>)  (source line information unavailable)
 69. PresentationFramework.dll#0!MS.Internal.Data.ClrBindingWorker.AttachDataItem()  (source line information unavailable)
 70. PresentationFramework.dll#0!System.Windows.Data.BindingExpression.Activate(item=<N/A>)  (source line information unavailable)
 71. PresentationFramework.dll#0!System.Windows.Data.BindingExpression.AttachToContext(attempt=<N/A>)  (source line information unavailable)
 72. PresentationFramework.dll#0!System.Windows.Data.BindingExpression.MS.Internal.Data.IDataBindEngineClient.AttachToContext(lastChance=<N/A>)  (source line information unavailable)
 73. PresentationFramework.dll#0!MS.Internal.Data.DataBindEngine+Task.Run(lastChance=<N/A>)  (source line information unavailable)
 74. PresentationFramework.dll#0!MS.Internal.Data.DataBindEngine.Run(arg=<N/A>)  (source line information unavailable)
 75. PresentationFramework.dll#0!MS.Internal.Data.DataBindEngine.OnLayoutUpdated(sender=<N/A>, e=<N/A>)  (source line information unavailable)
 76. PresentationCore.dll#0!System.Windows.ContextLayoutManager.fireLayoutUpdateEvent()  (source line information unavailable)
 77. PresentationCore.dll#0!System.Windows.ContextLayoutManager.UpdateLayout()  (source line information unavailable)
 78. PresentationCore.dll#0!System.Windows.ContextLayoutManager.UpdateLayoutCallback(arg=<N/A>)  (source line information unavailable)
 79. PresentationCore.dll#0!System.Windows.Media.MediaContext.FireInvokeOnRenderCallbacks()  (source line information unavailable)
 80. PresentationCore.dll#0!System.Windows.Media.MediaContext.RenderMessageHandlerCore(resizedCompositionTarget=<N/A>)  (source line information unavailable)
 81. PresentationCore.dll#0!System.Windows.Media.MediaContext.RenderMessageHandler(resizedCompositionTarget=<N/A>)  (source line information unavailable)
 82. WindowsBase.dll#0!System.Windows.Threading.ExceptionWrapper.InternalRealCall(callback=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 83. WindowsBase.dll#0!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(source=System.Windows.Threading.Dispatcher, method=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<null>)  (source line information unavailable)
 84. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WrappedInvoke(callback=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<N/A>)  (source line information unavailable)
 85. WindowsBase.dll#0!System.Windows.Threading.DispatcherOperation.InvokeImpl()  (source line information unavailable)
 86. mscorlib.dll#0!System.Threading.ExecutionContext.runTryCode(userData=<N/A>)  (source line information unavailable)
 87. mscorlib.dll#0!System.Threading.ExecutionContext.Run(executionContext=<N/A>, callback=<N/A>, state=<N/A>, ignoreSyncCtx=<N/A>)  (source line information unavailable)
 88. mscorlib.dll#0!System.Threading.ExecutionContext.Run(executionContext=<N/A>, callback=<N/A>, state=<N/A>)  (source line information unavailable)
 89. WindowsBase.dll#0!System.Windows.Threading.DispatcherOperation.Invoke()  (source line information unavailable)
 90. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.ProcessQueue()  (source line information unavailable)
 91. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WndProcHook(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 92. WindowsBase.dll#0!MS.Win32.HwndWrapper.WndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
 93. WindowsBase.dll#0!MS.Win32.HwndSubclass.DispatcherCallbackOperation(o=<N/A>)  (source line information unavailable)
 94. WindowsBase.dll#0!System.Windows.Threading.ExceptionWrapper.InternalRealCall(callback=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 95. WindowsBase.dll#0!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(source=System.Windows.Threading.Dispatcher, method=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<null>)  (source line information unavailable)
 96. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.WrappedInvoke(callback=<N/A>, args=<N/A>, numArgs=<N/A>, catchHandler=<N/A>)  (source line information unavailable)
 97. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.InvokeImpl(priority=<N/A>, timeout=<N/A>, method=<N/A>, args=<N/A>, numArgs=<N/A>)  (source line information unavailable)
 98. WindowsBase.dll#0!MS.Win32.HwndSubclass.SubclassWndProc(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>)  (source line information unavailable)
    [IL Method without Metadata]
    [Internal Frame, 'M-->U']
    [IL Method without Metadata]
 99. WindowsBase.dll#0!System.Windows.Threading.Dispatcher.PushFrameImpl(frame=<N/A>)  (source line information unavailable)
 100. PresentationFramework.dll#0!System.Windows.Application.RunInternal(window=<N/A>)  (source line information unavailable)
 101. PresentationFramework.dll#0!System.Windows.Application.Run()  (source line information unavailable)
 102. MyProgram.exe#0!XamlGeneratedNamespace.GeneratedApplication.Main()  (source line information unavailable)

It seems that someone is holding the WriteLock but never release - but how can I check who's holding that? I've paste the whole stacktrace I've got here, is someone to give me some hits on the root cause, like what's HwndSubclass and why it happens repeatedly in the stacktrace with the change of IsActive and WindowState property?

Please add comments if you need more information.

Jeffrey Zhao
  • 4,923
  • 4
  • 30
  • 52
  • What do you do in IsActive's getter? It should be as simple as possible (no locks, no IO operations, no heavy calculations). Also, ensure that the ProppertyChanged event is fired in a UI thread. Although, starting from .NET Framework 4.0 (or earlier?) it is dispatched to a UI thread automatically. – Sergii Vashchyshchuk Jun 10 '14 at 05:46
  • @SergiiVashchyshchuk The getter as simple as automatic property. Ensuring PropertyChanged event is fired in UI thread is the workaround I've applied, but I want to know the root cause of the issue since notify property change from background thread is a acceptable pattern now. The current weird thing is why the GUI thread would get blocked by the reader lock when it's holding the writer lock itself. – Jeffrey Zhao Jun 10 '14 at 06:31
  • It also uses this ReadLock when it subscribes to PropetyChanged event. It subscribes to PropetyChanged event when you bind your model to the grid. How do you bind your collection of models to the grid? What type of the collection? How do you populate it? – Sergii Vashchyshchuk Jun 10 '14 at 23:34
  • @SergiiVashchyshchuk It uses WriteLock to maintain event handlers. Binding to grid and updating models are nothing special. You could assume it's just simple array and simple property set with firing PropertyChanged event. – Jeffrey Zhao Jun 11 '14 at 02:07
  • I hope you did profile it for memory issues, if not then I could suggest to go for memory profiling and see if there is something interesting to be fixed. See this example on Wikipedia about memory leak demonstrating a similar issue with elevator software. http://en.wikipedia.org/wiki/Memory_leak#An_example_of_memory_leak, just to mention about software aging here http://en.wikipedia.org/wiki/Software_aging – pushpraj Jun 11 '14 at 04:10
  • @pushpraj Thanks for mentioning the case. I'll take a look to it. We do have memory profiling regularly, especially before each release, and this issue is raised again from with the new QA release. – Jeffrey Zhao Jun 11 '14 at 06:43
  • So, you want to say you bind a fixed collection with items once, do not modify it, only update properties on items forcing them to generate PropertyChanged event? Another questions, how big is your grid (in terms of columns)? How many properties on items do you have and how ofthen do you modify them? – Sergii Vashchyshchuk Jun 11 '14 at 07:07
  • @SergiiVashchyshchuk ~20 grids. Each grid has 1~20K rows and each row has 100~200 columns. The updating is rather frequent since the issue normally happens when the whole layout is being loaded from scratch. – Jeffrey Zhao Jun 11 '14 at 07:16
  • You can find the HwndSubclass here: http://referencesource.microsoft.com/#WindowsBase/src/Shared/MS/Win32/HwndSubclass.cs#7ce3eb4c026c666f which seems to be helping with Win32. Also the comments say it's not thread safe either. Perhaps looking it through could help you. – Zache Jun 11 '14 at 07:28
  • @Zache Thanks. I'm also checking the code from there. Concurrency is not the root cause. I think I've solve the issue and would answer my own question after having more verification. – Jeffrey Zhao Jun 11 '14 at 13:12
  • @JeffreyZhao Do you have a project that it can be reproduce? I would like to take a look. – 123 456 789 0 Jun 12 '14 at 15:39
  • @lll No, I was not able to write a simple app that produce the issue. Now I know the cause so maybe I could try again, but since I still cannot 100% explain the reason, it's still very likely to fail again. Anyway, it's an interesting issue worth investigating. I'll post further information here. – Jeffrey Zhao Jun 13 '14 at 02:21
  • From call stack it looks like you have two-way binding on IsActive property in your ViewModel, and it seems to cause collision. Can you change this binding to OneWayToSource or OneWay (depending on logic behind this property)? – Woodman Jun 13 '14 at 06:45
  • @Woodman Why does Two-Way binding cause the issue? In `WindowViewModel.IsActive` there's a check to guarantee `PropertyChanged` event fires only when the new value doesn't equals to the current value. I think the dependency property in control does the same thing. – Jeffrey Zhao Jun 13 '14 at 07:21

3 Answers3

4

I've finally fix the issue after digging into the source code of almost all the related components. Thanks to the fantastic Reference Source web site, the comments in source code help a lot comparing to the de-compiled result from ILSpy.

What's PropertyChangedEventManager?

PropertyChangedEventManager (source code here) is a component that handles the ProperyChanged event handlers and notifications in a concurrent environment. In the other words, it's thread-safe. Internally it uses a ReaderWriterLock to keep the thread safety. The writer lock would be acquired when the event handlers are being changed, and the reader lock would be acquired when there're PropertyChanged event notifications.

PropertyChangedEventManager is normally used by WPF controls. When we attach/detach the view model to the control, we're adding/removing PropertyChanged event handlers. I was always wondering who's holding the writer lock which blocks the reader lock (get_ReadLock), but actually it's the GUI thread itself.

Yes it sounds weird but it is just inside PrivateAddListener (source code here) as the stacktrace shows:

...
[IL Method without Metadata]
 66. WindowsBase.dll#0!System.ComponentModel.PropertyChangedEventManager.PrivateAddListener(source=<N/A>, listener=<N/A>, propertyName=<N/A>)  (source line information unavailable)
 67. PresentationFramework.dll#0!MS.Internal.Data.PropertyPathWorker.ReplaceItem(k=<N/A>, newO=<N/A>, parent=<N/A>)  (source line information unavailable)
...

BTW, I was always told that we should fire PropertyChanged event in an UI-bound object from background thread, but it's not the case since .NET 4. PropertyChangedEventManager is designed to be used in a concurrent environment. The exclusive (writer) lock would be used only when the model is binding to the GUI control, and the PropertyChanged event could be trigger from multiple background threads concurrently. We don't need to marshal everything to the GUI thread manually.

Actually it's a very important pattern to update models in background thread, and sometimes that's the only acceptable approach. Please consider the case that when we have multiple GUI/STA threads to improve the responsiveness of the application. We could bind the same instance to the controls in different GUI thread. When the model is changed, we simply cannot marshal the PropertyChanged notification into any one of them. Cross threading notification is inevitable.

What's SubclassWndProc?

HwndSubclass.SubclassWndProc (source code here) is the entry point of managed code to handle window messages. It's called by native code so we could always find [IL Method without Metadata] and [Internal Frame, 'M-->U'] before it in the stacktrace.

The weird thing is, why there would be several SubclassWndProc calls in the stacktrace? Shouldn't the window messages be processed one after another, separately? To answer the question, we need to check the code of the methods appears repeatedly in the stacktrace:

...
 55. PresentationFramework.dll#0!System.Windows.Window.HandleActivate(windowActivated=<N/A>)  (source line information unavailable)
 56. PresentationFramework.dll#0!System.Windows.Window.WmActivate(wParam=<N/A>)  (source line information unavailable)
 57. PresentationFramework.dll#0!System.Windows.Window.WindowFilterMessage(hwnd=<N/A>, msg=<N/A>, wParam=<N/A>, lParam=<N/A>, handled=<N/A>)  (source line information unavailable)
...

From the source code I noticed that these methods are dealing with WM_ACTIVATE message (OK we could also tell it from the name). It's a special message as decribed in MSDN:

Sent to both the window being activated and the window being deactivated. If the windows use the same input queue, the message is sent synchronously, first to the window procedure of the top-level window being deactivated, then to the window procedure of the top-level window being activated. If the windows use different input queues, the message is sent asynchronously, so the window is activated immediately.

Since the message is sent synchronously in GUI thread, it would be processed immediately without finishing the current one. That's the reason we could find recursive calls in the stacktrace.

It also explains why WindowViewMode.IsActive is set multiple times:

  1. in #47 - WindowViewModel.set_IsActive(value=True)
  2. in #25 - WindowViewModel.set_IsActive(value=False)
  3. in #3 - WindowViewModel.set_IsActive(value=True)

Since it would be "deactivated" and "activated" again.

Root cause and solution

In WindowViewModel we have an IsActive property which is synchronized with Window.IsActive property. Please note it's not two-way binding since the property is read-only. When the window is activated, WindowViewModel.IsActive property would be set and trigger the PropertyChanged event. Since the WPF control has been hooked up with the view model, so the internal logic is executed.

I'm not clear what the logic is in detail (it's [IL Method without Metadata]) but unfortunately it produces a new WM_ACTIVATE message. This happens again and again and finally stop the GUI thread.

After making sure we're not using WindowViewModel.IsActive in binding, I changed it to the IsActive() method. We don't need to trigger the PropertyChanged event since it's not a property anymore.

I also left a comment said that if we do require IsActive to be a property, we need to make sure the PropertyChanged event is triggered inside Dispatcher.BeginInvoke, even it's already in the GUI thread. We need to make sure the next WM_ACTIVATE message is produced asynchronously.

One thing I cannot explain

But I still cannot explain why the ReaderWriterLock would block when we're acquiring the reader lock for the third or fourth time. I do think we have deeper recursive PropertyChanged notifications so the reader lock would be acquired more times than the current case. But every time we run into this issue, we always have the IsActive property in the stack trace.

Is there any special protection in ReaderWriterLock or WPF or even in the OS?

Jeffrey Zhao
  • 4,923
  • 4
  • 30
  • 52
  • 2
    According to MSDN ReaderWriterLock has interesting implementation: 1. A thread can hold a reader lock or a writer lock, but not both at the same time 2. ReaderWriterLock alternates between a collection of readers, and one writer 3. While a thread in the writer queue is waiting for active reader locks to be released, threads requesting new reader locks accumulate in the reader queue. And according to call stack we should've obtained Write lock at #66, and most probably were trying to obtain ReadLock on same PropertyChangedEventManager on top of the stack... – Woodman Jun 13 '14 at 11:09
  • @Woodman The same thread could acquire the reader lock when it has acquired the writer lock. We can verify that by simple code. From the stacktrace the first two `get_ReadLock` is fine but it stopped at the third one. – Jeffrey Zhao Jun 13 '14 at 13:24
  • 2
    You are supposed to call `DowngradeFromWriterLock` if you want to take a Reader lock when you currently have a Writer lock; or call `UpgradeToWriterLock` if you have a reader lock and want take a Writer lock and currently have a Reader lock – Peter Ritchie Jun 13 '14 at 13:41
  • @PeterRitchie `UpgradeToWriterLock` is for releasing a reader lock in order to acquire the writer lock. It would return a `LockCookie` to pass to `DowngradeToWriterLock`. When you've acquired a writer lock, you're free to call `AcquireReaderLock`, and you'll still have the writer lock. – Jeffrey Zhao Jun 13 '14 at 13:55
  • 2
    @JeffreyZhao No, the documentation for ReaderWriterLock is very explicit: "A thread can hold a reader lock or a writer lock, but not both at the same time" see http://msdn.microsoft.com/en-us/library/system.threading.readerwriterlock(v=vs.110).aspx – Peter Ritchie Jun 13 '14 at 14:05
  • @JeffreyZhao If you have a Reader lock and you need to write, you must upgrade to Writer, perform the write, then downgrade to Reader and then unlock if you no longer need to read. – Peter Ritchie Jun 13 '14 at 14:06
  • @PeterRitchie You're right about "upgrading from reader lock to writer lock", but we don't need that in current case. – Jeffrey Zhao Jun 13 '14 at 14:11
  • @PeterRitchie When the thread is holding the writer lock, you could call `AcquireReaderLock` but it still has the writer lock. That's what the documentation says. Regarding the `DowngradeFromWriterLock` method, do you notice that it requires a `LockCookie` get from `UpgradeToWriterLock`? Please write some code to test the API. – Jeffrey Zhao Jun 13 '14 at 14:17
  • @JeffreyZhao The docs for `AcquireReaderLock` also state that "If the current thread already has the writer lock, **no reader lock is acquired**". I don't know what else to tell you, the documentation states "A thread can hold a reader lock or a writer lock, but not both at the same time". A single test does not disprove the documentation. If having both Reader and Writer at the same time is "unsupported" it's likely that state is "undefined"--which means sometimes it won't fail and sometimes it will. "sometimes it won't fail" doesn't make it the right thing to do. – Peter Ritchie Jun 13 '14 at 14:33
  • @PeterRitchie Calling `AcquireReaderLock` when the current thread is holding the writer lock doesn't get you the reader lock, it just happens **nothing**. That's what the "**no reader lock is acquired**" is saying. – Jeffrey Zhao Jun 13 '14 at 14:40
  • Well, it's not that "nothing" happens--you don't get a Reader lock and your **Writer lock count goes up by one**. (i.e. you have to *release the writer lock* **not** release the reader lock afterwards). But, I don't recommend doing it that way because you have to know you have a writer lock and call the correct `Release` method--which really kinda defeats the purpose. – Peter Ritchie Jun 13 '14 at 17:08
  • @PeterRitchie Have you checked out [the document of `ReleaseReaderLock` method](http://msdn.microsoft.com/en-us/library/system.threading.readerwriterlock.releasereaderlock.aspx)? It tells us clearly "If a thread has the writer lock, calling `ReleaseReaderLock` has the same effect as calling `ReleaseWriterLock`." We could and should always use the correct `Release` method for corresponding `Acquire` method. – Jeffrey Zhao Jun 14 '14 at 02:26
2

There's a thread other than GUI thread filling data to models bind to the grid and triggers INotifyPropertyChanged.PropertyChanged event.

Don't do that. There is no safe way to update a UI-bound object from a background thread. At best it may work for awhile, then break horribly again for no apparent reason.

If you are making I/O bound calls, don't use a background thread. Instead use the async/await syntax to avoid blocking and handle everything on the UI thread.

If you are CPU bound then you need to marshall the updates back onto the UI thread. I would do this in batches rather than per-object as the thread marshalling can be expensive.

Jonathan Allen
  • 68,373
  • 70
  • 259
  • 447
  • 1
    I typically like to use Rx.Net for doing CPU bound work on batches of objects and then marshalling to the UI thread. It has lots of methods that help a lot. – Aron Jun 13 '14 at 03:18
  • 4
    I don't agree with you. In theory there might be several GUI/STA threads in the process and the same models could be bind to multiple GUI components in different GUI thread. Since .NET 4.0 it's a normal pattern to trigger `PropertyChanged` event from background thread. The `PropertyChangedEventManager` handles the cross thread notification well. Marshalling everything (batch required) into GUI brings complexity and normally it breaks the simple design also. – Jeffrey Zhao Jun 13 '14 at 03:44
0

It looks very suspicious that within same callstack you are changing value of IsActive property 3 times:

  1. in #47 - WindowViewModel.set_IsActive(value=True)
  2. in #25 - WindowViewModel.set_IsActive(value=False)
  3. in #3 - WindowViewModel.set_IsActive(value=True)

All three times setter was invoked as result of Dependency property change in #54, #32 and #10 respectively. Can you check your sources and explain two things:

  1. How can first change of WindowViewModel.IsActive with value True can lead to subsequent change of same property to False? (I can assume that False value goes to another instance of WindowViewModel class, but I'm not sure, without seen sources)
  2. and most important how this assignment of False value into IsActive property in turn leads into assignment of True value to same property?

That were the reasons why I've assumed that you've used TwoWay binding for IsActive property, and suggested to replace it with OneWay/OneWayToSource binding in my comment, as such change would eliminate possible interference between UI and ViewModel layers here.

Woodman
  • 1,108
  • 9
  • 11
  • Actually it's an One-Way binding from `Window.IsActive` to `WindowView.IsActive`. We cannot build two-way binding for `IsActive` since it's readonly property. BTW, I've fixed the issue and answer the question myself. Thanks for your help. – Jeffrey Zhao Jun 13 '14 at 10:50