How does a thread that launches a blocking I/O request under TPL return immediately?

Question

I would like to preface this question with the following:

1. I'm familiar with the IAsyncStateMachine implementation that the await keyword in C# generates.

2. My question is not about the basic flow of control that ensures when you use the async and await keywords.

Assumption A

The default threading behaviour in any threading environment, whether it be at the Windows operating system level or in POSIX systems or in the .NET thread pool, has been that when a thread makes a request for an I/O bound operation, say for a disk read, it issues the request to the disk device driver and enters a waiting state. Of course, I am glossing over the details because they are not of moment to our discussion.

Importantly, that thread can do nothing useful until it is unblocked by an interrupt from the device driver notifying it of completion. During this time, the thread remains on the wait queue and cannot be re-used for any other work.

I would first like a confirmation of the above description.

Assumption B

Secondly, even with the introduction of TPL, and its enhancements done in v4.5 of the .NET framework, and with the language level support for asynchronous operations involving tasks, this default behaviour described in Assumption A has not changed.

Question

Then, I'm at a loss trying to reconcile Assumptions A and B with the claim that suddenly emerged in all TPL literature that:

When the, say, main thread, starts this request for this I/O bound work, it immediately returns and continues executing the rest of the queued up messages in the message pump.

Well, what makes that thread return back to do other work? Isn't that thread supposed to be in the waiting state in the wait queue?

You might be tempted to reply that the code in the state machine launches the task awaiter and if the awaiter hasn't completed, the main thread returns.

That beggars the question -- what thread does the awaiter run on?

And the answer that springs up to mind is: whatever the implementation of the method be, of whose task it is awaiting.

That drives us down the rabbit hole further until we reach the last of such implementations that actually delivers the I/O request.

Where is that part of the source code in the .NET framework that changes this underlying fundamental mechanism about how threads work?

Side Note

While some blocking asynchronous methods such as WebClient.DownloadDataTaskAsync, if one were to follow their code through their (the method's and not one's own) oval tract into their intestines, one would see that they ultimately either execute the download synchronously, blocking the current thread if the operation was requested to be performed synchronously (Task.RunSynchronously()) or if requested asynchronously, they offload the blocking I/O bound call to a thread pool thread using the Asynchronous Programming Model (APM) Begin and End methods.

This surely will cause the main thread to return immediately because it just offloaded blocking I/O work to a thread pool thread, thereby adding approximately diddlysquat to the application's scalability.

But this was a case where, within the bowels of the beast, the work was secretly offloaded to a thread pool thread. In the case of an API that doesn't do that, say an API that looks like this:

public async Task<string> GetDataAsync()
{
  var tcs = new TaskCompletionSource<string>();

  // If GetDataInternalAsync makes the network request
  // on the same thread as the calling thread, it will block, right?
  // How then do they claim that the thread will return immediately?
  // If you look inside the state machine, it just asks the TaskAwaiter
  // if it completed the task, and if it hasn't it registers a continuation
  // and comes back. But that implies that the awaiter is on another thread
  // and that thread is happily sleeping until it gets a kick in the butt
  // from a wait handle, right?
  // So, the only way would be to delegate the making of the request
  // to a thread pool thread, in which case, we have not really improved
  // scalability but only improved responsiveness of the main/UI thread
  var s = await GetDataInternalAsync();

  tcs.SetResult(s); // omitting SetException and 
                    // cancellation for the sake of brevity

  return tcs.Task;
}

Please be gentle with me if my question appears to be nonsensical. The extent of knowledge of things in almost all matters is limited. I am just learning anything.

Answer 1

When you are talking about an async I/O operation, the truth, as pointed out here by Stephen Cleary (http://blog.stephencleary.com/2013/11/there-is-no-thread.html) is that there is no thread. An async I/O operation is completed at a lower level than the threading model. It generally occurs within interrupt handler routines. Therefore, there is no I/O thread handling the request.

You ask how a thread that launches a blocking I/O request returns immediately. The answer is because an I/O request is not at its core actually blocking. You could block a thread such that you are intentionally saying not to do anything else until that I/O request finishes, but it was never the I/O that was blocking, it was the thread deciding to spin (or possibly yield its time slice).

The thread returns immediately because nothing has to sit there polling or querying the I/O operation. That is the core of true asynchronicity. An I/O request is made, and ultimately the completion bubbles up from an ISR. Yes, this may bubble up into the thread pool to set the task completion, but that happens in a nearly imperceptible amount of time. The work itself never had to be ran on a thread. The request itself may have been issued from a thread, but as it is an asynchronous request, the thread can immediately return.

Let's forget C# for a moment. Lets say I am writing some embedded code and I request data from a SPI bus. I send the request, continue my main loop, and when the SPI data is ready, an ISR is triggered. My main loop resumes immediately precisely because my request is asynchronous. All it has to do is push some data into a shift register and continue on. When data is ready for me to read back, an interrupt triggers. This is not running on a thread. It may interrupt a thread to complete the ISR, but you could not say that it actually ran on that thread. Just because its C#, this process is not ultimately any different.

Similarly, lets say I want to transfer data over USB. I place the data in a DMA location, set a flag to tell the bus to transfer my URB, and then immediately return. When I get a response back it also is moved into memory, an interrupt occurs and sets a flag to let the system know hey, heres a packet of data sitting in a buffer for you.

So once again, I/O is never truly blocking. It could appear to block, but that is not what is happening at the low level. It is higher level processes that may decide that an I/O operation has to happen synchronously with some other code. This is not to say of course that I/O is instant. Just that the CPU is not stuck doing work to service the I/O. It COULD block if implemented that way, and this COULD involve threads. But that is not how async I/O is implemented.