Wpf application with multiple STA threads still blocks user interface

Question

A while ago we added Python scripting to a Wpf application using IronPython. At first it was only 'slave' in the sense that a script was invoked by a button click for instance and then just ran to completion returning control to Wpf. Later on we added 'master' scripting: the script runs in it's own thread, and controls the rest of the application. That was quite challenging but after a while and with help of existing SO content we got it working, seemingly. Never really used it though, until now, and unfortunately it turns out it does not work properly. Core cause is that although there are two seperate STA threads (the main Wpf one and one for the script), and hence two different Dispatcher instances, the main thread seems to get blocked because the script thread is in a loop waiting for the main thread to complete (in response to a button click processed on the script thread and starting events on the main thread). The whole point of using two threads with seperate ui windows was of course this wouldn't happen. What is going on?

update It is reproducable with minimal code, so I'm linking to that instead of posting pseudo-code here. While creating the code I found that when the window created by the script thread is not embedded (set MainWindow.hostedWin = false) the deadlock does not occur and everything behaves as expected.

in response to comments So there are 3 threads of concern coming into play. Let's call them Python, Ui and Process. Python starts Process and waits for it to complete. Process calls Invoke on Ui. Which shouldn't be doing anything at that point: after all, it's Python that is blocking, not Ui, and the whole point of this construction is that Ui shouldn't have to interact with Python. Well, except that it does somehow. Which is the culprit. In the deadlock, Ui sits at PresentationFramework.dll!System.Windows.Interop.HwndHost.OnWindowPositionChanged(System.Windows.Rect rcBoundingBox) + 0x82 bytes and Process sits at WindowsBase.dll!System.Windows.Threading.DispatcherOperation.DispatcherOperationEvent.WaitOne() + 0x2f bytes and Python is just at Thread.Sleep.

What is going on here, and how to fix it?

Answer 1

I'll keep it short, very few odds that this answer is going to make you happy. It is a three-way deadlock. The most severe one in the interaction between the main thread and PythonThread. This deadlock occurs in the Windows kernel, the NtUserSetWindowPos() call cannot progress. It is blocked, waiting for the WM_LBUTTONUP callback notification on the PythonThread to finish running.

This deadlock is caused by your WpfHwndEmbedHost hack. Turning a top-level window owned by another thread or process into a child window is an appcompat feature that was meant to support Windows 3.x programs. A Windows version that did not yet support threads and where having one task embedding another task's window wasn't a problem. A WPF window isn't exactly much like such a window, to put it mildly. Otherwise a well-known troublemaker, for one the reason that embedding Acrobat Reader in a browser window works so very poorly. Not turning on the WS_CHILD style flag ought to bring relief, but WPF isn't happy about that. Simply setting hostedWin to false solves the problem.

The other deadlock is the one I warned you about, the interaction between the main thread and the ProcessThread. Dispatcher.Invoke() is dangerous, it deadlocks because the main thread is stuck in the kernel. Using Dispatcher.BeginInvoke() solves the problem. Partly, you still have the main thread go catatonic for 5 seconds.

The most severe problem is the kernel lock, that's going to bite in many other ways. You are going to have to keep it a separate window to avoid it. Not good news, I'm sure.

Answer 2

This is a long shot but you might have to implement your own SynchronizationContext to achieve this.

As far as I understand from Andrew Nosenko's answer and his references, it seem that CLR has its own mind about the message pump for the UI thread and it's not actually possible to run two UI threads under one window (BTW, I was able to replicate the issue without IronPython, that seems to be irrelevant here)

Main reference is from cbrumme's WebLog 'Apartments and Pumping in the CLR'

I keep saying that managed blocking will perform “some pumping” when called on an STA thread. Wouldn’t it be great to know exactly what will get pumped? Unfortunately, pumping is a black art which is beyond mortal comprehension. On Win2000 and up, we simply delegate to OLE32’s CoWaitForMultipleHandles service. And before we wrote the initial cut of our pumping code for NT4 and Win9X, I thought I would glance through CoWaitForMultipleHandles to see how it is done. It is many, many pages of complex code. And it uses special flags and APIs that aren’t even available on Win9X.

I must admit I am a little out of my depth here, and might be missing the point completely, so apologies upfront if this might not be an answer to the question at all (Nevertheless it's been a good experience for me, that's for sure).

I have tried using Andrew Nosenko's SynchronizationContext implementation to come up with an example unfortunately without success. Hope it helps you and good luck!

Answer 3

I have delt with similiar problems in my applications where i invoked some UI updates from my heavy load threads and had the same result the UI blocked the thread. I made a solution which i now use in every application though you would need to apply it to your application it works like this:

Apart from the thread you do your work in (and the UI thread) you also need to create another thread, this one will take data from a stack and send it to the UI thread.

Essentialy when you want your UI get a update by your work thread you save the result of your work thread into a List if its more complex data then you would need to create a struct and save all the current data from the thread into the struct and add it to the List (adding data to to the list doesnt require invoke).

Now your second threads runs in a loop and checks at a certain interval if there is something in the List if there is he adds the list elements to your UI.

Here is a sample of how it should work

using System;
using System.Windows;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Diagnostics;
using System.Threading;

namespace nonblock
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private ListBox l1;
        private ListBox l2;

        private Thread workThread;
        private Thread nonBlockThread;

        private List<TwoNumbers> list;
        private void Form1_Load(object sender, EventArgs e)
        {
            this.Size = new Size(500,500);
            this.FormClosing += (ss, ee) =>
            {
                workThread.Abort();
                nonBlockThread.Abort();
            };
            l1 = new ListBox();
            l1.Dock = DockStyle.Left;
            l2 = new ListBox();
            l2.Dock = DockStyle.Right;
            list = new List<TwoNumbers>();

            this.Controls.Add(l1);
            this.Controls.Add(l2);

            workThread = new Thread(work);
            workThread.Start();

            nonBlockThread = new Thread(update);
            nonBlockThread.Start();
        }

        private void work()
        {
            int a = 0;
            int b = 0;
            int counter = 0;
            Random r = new Random();
            while (true)
            {
                a += r.Next();
                b += r.Next();
                counter++;
                if (counter % 10 == 0)
                    list.Add(new TwoNumbers(a, b));
                Thread.Sleep(40);
            }
        }

        private void update()
        {
            while (true)
            {
                if (list.Count > 0)
                {
                    for (int a = 0; a < list.Count; a++)
                    {
                        l1.Invoke((MethodInvoker)(() => l1.Items.Add(list[0].n1)));
                        l2.Invoke((MethodInvoker)(() => l2.Items.Add(list[0].n2)));
                        list.RemoveAt(0);
                    }
                }
                Thread.Sleep(1000);
            }
        }

        public class TwoNumbers
        {
            public int n1 { get; set; }
            public int n2 { get; set; }

            public TwoNumbers(int a, int b)
            {
                n1 = a;
                n2 = b;
            }
        }
    }
}