Console.In.ReadLine (redirected StdIn) freezes all other threads

Question

I'm connecting a legacy application (written in COBOL) with our .NET new ones using pipes. The idea is simple: The legacy program (my ERP Menu) writes some parameters upon the stream, the .NET app reads it thru Console.In stream and starts a new thread opening the screen requested. Here's a snippet from the .NET side of how the idea works:

<STAThread(), LoaderOptimization(LoaderOptimization.MultiDomain)>
Shared Sub Main()
    If Environment.GetCommandLineArgs(1) = "PIPE"
       While True
          Dim pipeParam as String
          Try
             pipeParam = Console.In.ReadLine()
          Catch e as Exception
             Exit Sub
          End Try

          ' Deal with parameters here

          If pipeParam = "END"
             Dim newThread as New Threading.Thread(Sub()
                                                       ' Create a new AppDomain and Loads the menu option, generally a Winforms form.
                                                   End Sub)
             newThread.Start()                 
          End If
       End While

    End If
End Sub

Everything was working fine and easy... until today. I deployed this solution in my client environment (Windows Server 2003), and it happened that none of the threads requested were being executed, except when the called process (COBOL) was being terminated (that is, the Console.In was being forcedly closed). From then on, all the requested winforms will start showing up and behaving as expected.

Digging this strange behaviour with logs, I found out that the threads were being normally executed until a point that a IDictionary.ContainsKey() statement was performed (or some other method that requires native code execution). At this point, the thread was freezing / sleeping.

If I limit the thread creation to, let say, three, it happens that every created thread hangs until the third one, that is, when Console.In.ReadLine is not executed anymore.

What should I try? Any advices?

Some more info: The closest direction I've found so far was the Hans Passant's answer in this question: Interface freezes in multi-threaded c# application (the .NET SystemEvents happens to appear in my debbuger thread list, but I couldn't solve my problem with the proposed solution).

UPDATED NEWS

I could solve this issue by waiting the sub-thread to finish load the Form. This "Is Ready" signal is passed through AppDomain.SetData() and AppDomain.GetData(). Somehow, after the form creation the sub-thread doesn't freezes anymore when the main one goes on Console.ReadLine. Though the problem is solved, I'm intrigued with this. I'm trying to reproduce this in a "simple as possible" test case.

Some More Details

• The entry-point .exe is compiled to 32-bit. All other libraries are 'AnyCpu'. The issue happens running on both 32-bits (my client) and 64-bits (my development) machines (both windows Server 2003).
• Updated Sub Main() attributes in the above snippet.
• Tried to put the Console.ReadLine in a worker thread. Didn't solved (see image below).
• The COBOL application won't freeze, because it is executed in a separate OS Process. The pipe happens to be my IPC approach. The COBOL application in this case only writes parameters and don't wait for response.
• The stack trace is in the image below (the thread PRX001235 is deserializing an xml config file before connecting to the database and before effectively loading the form - in this case, seems to be still in managed code - sometimes the PRX001235 thread will freeze in native code when trying to connect to the database):

Answer 1

First off, you are doing something very unusual, so unusual outcomes are not unexpected. Secondly, what you are doing is strictly verboten in the Windows SDK docs. Which dictates that a thread that created a single-threaded apartment is never allowed to make a blocking call.

Clearly your worker threads are blocked on a lock somewhere inside the bowels of the operating system, a lock taken by the main thread of your program. It would help a lot to see the call stack of one of those blocked threads to identify the possible lock. That requires enabling unmanaged code debugging so you can see the unmanaged stack frames and enabling the Microsoft Symbol Server so you get debugging symbols for the Windows code and can make sense out of the stack trace.

Not having one to look at, I'll have to speculate:

• Console.ReadLine() itself takes an internal lock that makes the console safe to use from multiple threads, serializing calls to Read(). Any thread that uses a Console method can hit that same lock. This is not likely the culprit, unlikely that these worker threads are also using the console.

• The strictly verboten angle is associated with the relevance of apartment threading, a COM feature. An STA is enabled by the [STAThread] attribute on your program's Main() method, or a Thread.SetApartmentState() call. An STA is required to use components that are fundamentally thread-unsafe, like windows, the clipboard, drag and drop, the shell dialogs like OpenFileDialog and many other COM coclasses, some of which you might not recognize since they were wrapped by .NET classes. An STA apartment ensures that these components are used in a thread-safe manner, a call to a method of such a component from a worker thread is automatically marshaled to the thread that created it. The exact equivalent to Control.Invoke(). Such marshaling requires that the thread pumps a message loop to dispatch the call on the right thread. And your main thread is not pumping when it is blocked on the Console.ReadLine() call. The worker thread will be stalled on the call until the main thread starts pumping again. Very high odds for deadlock, albeit that you don't actually deadlock completely because the ReadLine() call ultimately completes. Notable is that the CLR avoids these kind of deadlocks, it pumps a message loop when you use the lock keyword, call WaitOne() on a synchronization object or call Thread.Join(), the common kind of blocking calls in .NET programming. It however doesn't do this for Console.ReadLine(), at least until the .NET 4.0 workaround as shown by Mark.

• A highly speculative one, triggered by your observation that you avoid the problem by waiting for the form to be created. You might have this problem on a 64-bit operating system and your EXE project has the Platform target setting set to "x86" instead of "AnyCPU". In that case, your program runs in the WOW64 emulation layer, the layer that allows a 64-bit operating system to execute 32-bit programs. The Windows window manager is a 64-bit sub-system, calls it makes to send notifications to a 32-bit window go through a thunk that switches bit-ness. There's a problem with the Form.Load event, triggered when the window manager delivers the WM_SHOWWINDOW message. It puts the emulation layer in a difficult state because that travels the 64-bit to 32-bit boundary several times. This is also the source of a very nasty exception swallowing problem, documented in this answer. The odds are very high that this code holds an emulator lock at the time the Load event fires. So calling Console.ReadLine() in the Load event is likely to be very troublesome, I'd expect this lock to be passed by any 32-bit worker thread as well when they make an api call. Highly speculative, but easy to test if you can change the Platform target to AnyCPU.

Not so sure if it is worth chasing the cause of this problem, given that the solution is so simple. Just don't call Console.ReadLine() on your main thread. Call it on a worker thread instead. This also stops your UI from freezing when the COBOL program is unresponsive. Do note however that you now must marshal yourself with Control.Begin/Invoke() if whatever you receive also updates your UI.

Answer 2

Comparing .NET 3.5/2.0 and .NET 4.0 with Reflector: .NET 4.0 always explicitly calls __ConsoleStream.WaitForAvailableConsoleInput(hFile) in __ConsoleStream.ReadFileNative and I can find no equivalent call in .NET 3.5/2.0.

The WaitForAvailableConsoleInput InternalCall checks for a pipe, and if so avoids waiting on it if it's got data available or if it's closed.

---

Just summarising what I understand is the current state of this issue: It can be worked around by ensuring the other threads (AppDomains) are pumping messages before allowing the main thread to continue to the ReadLine call.

I think that can be a (nearly) final answer, as it means there are Windows messages going on during the server's version of ReadLine and that's enough for me to know the pumping is needed. This comes down to Windows documentation needing to mention where Windows messaging is needed/used.

Answer 3

have you tried setting these threads to background threads by setting Thread.IsBackground = true