13

Blocking threads is considered a bad practice for 2 main reasons:

  1. Threads cost memory.
  2. Threads cost processing time via context switches.

Here are my difficulties with those reasons:

  1. Non-blocking, async code should also cost pretty much the same amount of memory, because the callstack should be saved somewhere right before executing he async call (the context is saved, after all). And if threads are significantly inefficient (memory-wise), why doesn't the OS/CLR offer a more light-weight version of threads (saving only the callstack's context and nothing else)? Wouldn't it be a much cleaner solution to the memory problem, instead of forcing us to re-architecture our programs in an asynchronous fashion (which is significantly more complex, harder to understand and maintain)?

  2. When a thread gets blocked, it is put into a waiting state by the OS. The OS won't context-switch to the sleeping thread. Since way over 95% of the thread's life cycle is spent on sleeping (assuming IO-bound apps here), the performance hit should be negligible, since the processing sections of the thread would probably not be pre-empted by the OS because they should run very fast, doing very little work. So performance-wise, I can't see a whole lot of benefit to a non-blocking approach either.

What am I missing here or why are those arguments flawed?

Winston Smith
  • 153
  • 1
  • 8
  • 1
    Not all threads are equal here. Blocking the UI thread is bad as it leaves the app unresponsive. – Brian Rasmussen Jan 14 '16 at 20:29
  • If you *want* to have single-threaded blocking code then you certainly *can*. Most modern applications prefer not to do that. Consider a background process in a desktop app where you still want to be able to interact with the UI (such as providing a progress meter on the process). Or a web application with high throughput but heavy back-end operations, where tying up a server thread to wait for an operation would drastically reduce the number of concurrent users possible. – David Jan 14 '16 at 20:32
  • Try to apply your arguments to both a desktop application with a responsive user interface and a web application that scales to many requests – Wouter de Kort Jan 14 '16 at 20:32
  • And also keep in mind that the objects to maintain the async state are likely in the area of 10s of KB, while a thread stack is often at least 1 MB. And that pulling a new item from the async work queue is likely a couple hundred CPU cycles while switching to another thread may be thousands (interrupt overhead and OS scheduler code). – Dark Falcon Jan 14 '16 at 20:36
  • 1
    It avoids your user interface from going catatonic. It was added to C# above all to give programmers a shot a creating WinRT programs, nowadays called UWP. Simple things like opening a file can only be done with an asynchronous method. – Hans Passant Jan 14 '16 at 20:40
  • 1
    @Dark Falcon, that's why I asked in my 1st point why doesn't the OS/CLR offer us a more lightweight version of threads, instead of forcing us to migrate our code to an async architecture? If threads are inefficient for whatever reason, fix those inefficiencies, but why throw the concept out of the window? – Winston Smith Jan 14 '16 at 20:42
  • Because it is not possible on current hardware to make threads very much lighter and still operate in a manner that works with blocking operations in a generalized fashion. There are lightweight threads in Windows (fibers), for example, but they are cooperative, which basically means you have to write your code just about like you do with `async` in order to make them work. – Dark Falcon Jan 14 '16 at 21:06
  • @WinstonSmith I think that's a very good question. Good job on your reasoning. Of course there is an answer and I think Eric Lippert is currently nailing it. – Sedat Kapanoglu Jan 14 '16 at 21:08
  • 1
    I found this thread to be relevant: https://stackoverflow.com/q/8546273/625332 – Dmitry Feb 10 '18 at 15:50

2 Answers2

17

Non-blocking, async code should also cost pretty much the same amount of memory, because the callstack should be saved somewhere right before executing he async call (the context is saved, after all).

The entire call stack is not saved when an await occurs. Why do you believe that the entire call stack needs to be saved? The call stack is the reification of continuation and the continuation of the awaited task is not the continuation of the await. The continuation of the await is on the stack.

Now, it may well be the case that when every asynchronous method in a given call stack has awaited, information equivalent to the call stack has been stored in the continuations of each task. But the memory burden of those continuations is garbage collected heap memory, not a block of a million bytes of committed stack memory. The continuation state size is order n in the size of the number of tasks; the burden of a thread is a million bytes whether you use it or not.

if threads are significantly inefficient (memory-wise), why doesn't the OS/CLR offer a more light-weight version of threads

The OS does. It offers fibers. Of course, fibers still have a stack, so that's maybe not better. You could have a thread with a small stack I suppose.

Wouldn't it be a much cleaner solution to the memory problem, instead of forcing us to re-architecture our programs in an asynchronous fashion

Suppose we made threads -- or for that matter, processes -- much cheaper. That still doesn't solve the problem of synchronizing access to shared memory.

For what it's worth, I think it would be great if processes were lighter weight. They're not.

Moreover, the question somewhat contradicts itself. You're doing work with threads, so you are already willing to take on the burden of managing asynchronous operations. A given thread must be able to tell another thread when it has produced the result that the first thread asked for. Threading already implies asynchrony, but asynchrony does not imply threading. Having an async architecture built in to the language, runtime and type system only benefits people who have the misfortune to have to write code that manages threads.

Since way over 95% of the thread's life cycle is spent on sleeping (assuming IO-bound apps here), the performance hit should be negligible, since the processing sections of the thread would probably not be pre-empted by the OS because they should run very fast, doing very little work.

Why would you hire a worker (thread) and pay their salary to sit by the mailbox (sleeping the thread) waiting for the mail to arrive (handling an IO message)? IO interrupts don't need a thread in the first place. IO interrupts exist in a world below the level of threads.

Don't hire a thread to wait on IO; let the operating system handle asynchronous IO operations. Hire threads to do insanely huge amounts of high latency CPU processing, and then assign one thread to each CPU you own.

Now we come to your question:

What are the benefits of async (non-blocking) code?

  • Not blocking the UI thread
  • Making it easier to write programs that live in a world with high latency
  • Making more efficient use of limited CPU resources

But let me rephrase the question using an analogy. You're running a delivery company. There are many orders coming in, many deliveries going out, and you cannot tell a customer that you will not take their delivery until every delivery before theirs is completed. Which is better:

  • hire fifty guys to take calls, pick up packages, schedule deliveries, and deliver packages, and then require that 46 of them be idle at all times or

  • hire four guys and make each of them really good at first, doing a little bit of work at a time, so that they are always responsive to customer requests, and second, really good at keeping a to-do list of jobs they need to do in the future

The latter seems like a better deal to me.

Eric Lippert
  • 647,829
  • 179
  • 1,238
  • 2,067
  • 1
    About synchronizing - the async/await model doesn't solve that either, because the continuation runs on a fresh thread from the thread pool – Winston Smith Jan 14 '16 at 20:56
  • @WinstonSmith: The continuation runs on a worker thread in ASP.NET and console apps, but it runs on the UI thread when awaited on the UI thread. But you're right, there are definitely issues to be concerned about here. The async architecture seeks to mitigate many of those difficulties though by representing "a value that will appear in the future" in the type system, rather than making you synchronize access to shared memory yourself. – Eric Lippert Jan 14 '16 at 20:58
  • 5
    And there's no built in support for fibers in the CLR. As for your last point, why would I care paying for that worker to sit by the mailbox, if he's so cheap? When he sleeps, he costs no money (CPU) and when he's awake, he works so fast that the context switch problem should be a non-issue. Of course, there's the memory problem, but honestly, it's not such a big deal either way. – Winston Smith Jan 14 '16 at 21:01
  • 1
    @WinstonSmith: OK, suppose for the sake of argument that threads -- or processes -- were made super cheap. That still doesn't mitigate the problem of *how do you write programs effectively that live in a world where high latency operations are common*? That's the problem that asynchronous architecture is designed to solve. Whether it solves it with a small number of heavy threads or a large number of light threads is not germane to the actual problem, which is: *is the code comprehensible?* – Eric Lippert Jan 14 '16 at 21:04
  • 1
    I was talking more in the way of scalability concerns. – Winston Smith Jan 14 '16 at 21:11
-3

You are messing multithreading and async concepts here.

Both your "difficulties" come from the assumption that each async method gets assigned a specialized thread on which it does the work. However, the state of affairs is quite opposite: each time an async operation needs to be executed, the CLR picks an idle (thus already created) thread from the threadpool and executes that method on the selected thread.

The core concept here is that async doesn't mean always creating new threads, it means scheduling the execution on existing threads so that no thread is sitting idle.

RePierre
  • 9,358
  • 2
  • 20
  • 37
  • Actually, an asynchronous operation *doesn't necessarily even represent work done by a thread at all*. That's just *one* type of asynchronous operation. Many asynchronous operations involve no threads being used at all. – Servy Jan 14 '16 at 21:09
  • @Servy, **there is always a thread** - the main thread of the process. The fact that many `async` operations get scheduled to execute on that thread without using a separate thread doesn't mean there is no thread. – RePierre Jan 15 '16 at 06:59
  • **No**. An asynchronous operation does not necessarily need *any* thread to do its work. Performing CPU bound operations is only *one* kind of operation. You don't need *any* thread, not even the main thread, when, for example, sending a network request, or delaying for a fixed period of time. Those are operations that *need no thread at all*. – Servy Jan 15 '16 at 13:37
  • @Servy, can you please provide an article that describes these kind of operations? I will edit my answer then. – RePierre Jan 15 '16 at 14:06
  • Describes what about them? You can read an intro tutorial on asynchronous programming if you're just looking for basic information on how asynchronous operations work. – Servy Jan 15 '16 at 14:13
  • 1
    That describes exactly how (some) `async` operations don't need a thread at all and what happens under the hood. Because the "intro tutorials" talk about **not blocking the current thread** and **using time slices from current synchronization context**. I haven't found any talking about no thread at all. – RePierre Jan 15 '16 at 14:41
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/100779/discussion-between-repierre-and-servy). – RePierre Jan 15 '16 at 14:52