366

Time and time again, I see it said that using async-await doesn't create any additional threads. That doesn't make sense because the only ways that a computer can appear to be doing more than 1 thing at a time is

  • Actually doing more than 1 thing at a time (executing in parallel, making use of multiple processors)
  • Simulating it by scheduling tasks and switching between them (do a little bit of A, a little bit of B, a little bit of A, etc.)

So if async-await does neither of those, then how can it make an application responsive? If there is only 1 thread, then calling any method means waiting for the method to complete before doing anything else, and the methods inside that method have to wait for the result before proceeding, and so forth.

Ms. Corlib
  • 4,993
  • 4
  • 12
  • 19
  • 27
    IO tasks are not CPU bound and thus do not require a thread. The main point of async is to not block threads during IO bound tasks. – juharr May 24 '16 at 16:55
  • I agree with you. A new process must be started. – jdweng May 24 '16 at 16:56
  • 28
    @jdweng: Nope, not at all. Even if it created new *threads*, that's very different from creating a new process. – Jon Skeet May 24 '16 at 16:57
  • 6
    Suppose you make a web request - in the period between sending the request and receiving the response, do you consider yourself to be "doing" anything? Your program doesn't have to run any code, and yet something useful is occurring. – Jon Skeet May 24 '16 at 16:58
  • 10
    If you understand callback-based asynchronous programming, then you understand how `await`/`async` works without creating any threads. – user253751 May 24 '16 at 21:18
  • 7
    It doesn't exactly *make* an application more responsive, but it does discourage you from blocking your threads, which is a common cause of nonresponsive applications. – Owen May 24 '16 at 22:18
  • @JonSkeet correct me if I'm wrong, but didn't I read in your book that async-await may or may not use a thread from the pool? I'm struggling to remember where I read it. – RubberDuck May 25 '16 at 01:37
  • 12
    @RubberDuck: Yes, it may use a thread from the thread pool for the continuation. But it's not starting a thread in the way that the OP imagines here - it's not like it says "Take this ordinary method, now run it in a separate thread - there, that's async." It's much subtler than that. – Jon Skeet May 25 '16 at 05:18
  • 2
    Sometimes a task just needs to wait, and that isn't a threading issue. Say Task A is waiting on Call A for 300 milliseconds. Task B comes and needs to execute Local B for 3 nanoseconds. Should Task B wait for Call A? Hopefully you went with "no". Await the return from Call A and allow Local B to sneak in. – Travis J May 25 '16 at 21:58

11 Answers11

411

Actually, async/await is not that magical. The full topic is quite broad but for a quick yet complete enough answer to your question I think we can manage.

Let's tackle a simple button click event in a Windows Forms application:

public async void button1_Click(object sender, EventArgs e)
{
    Console.WriteLine("before awaiting");
    await GetSomethingAsync();
    Console.WriteLine("after awaiting");
}

I'm going to explicitly not talk about whatever it is GetSomethingAsync is returning for now. Let's just say this is something that will complete after, say, 2 seconds.

In a traditional, non-asynchronous, world, your button click event handler would look something like this:

public void button1_Click(object sender, EventArgs e)
{
    Console.WriteLine("before waiting");
    DoSomethingThatTakes2Seconds();
    Console.WriteLine("after waiting");
}

When you click the button in the form, the application will appear to freeze for around 2 seconds, while we wait for this method to complete. What happens is that the "message pump", basically a loop, is blocked.

This loop continuously asks windows "Has anyone done something, like moved the mouse, clicked on something? Do I need to repaint something? If so, tell me!" and then processes that "something". This loop got a message that the user clicked on "button1" (or the equivalent type of message from Windows), and ended up calling our button1_Click method above. Until this method returns, this loop is now stuck waiting. This takes 2 seconds and during this, no messages are being processed.

Most things that deal with windows are done using messages, which means that if the message loop stops pumping messages, even for just a second, it is quickly noticeable by the user. For instance, if you move notepad or any other program on top of your own program, and then away again, a flurry of paint messages are sent to your program indicating which region of the window that now suddenly became visible again. If the message loop that processes these messages is waiting for something, blocked, then no painting is done.

So, if in the first example, async/await doesn't create new threads, how does it do it?

Well, what happens is that your method is split into two. This is one of those broad topic type of things so I won't go into too much detail but suffice to say the method is split into these two things:

  1. All the code leading up to await, including the call to GetSomethingAsync
  2. All the code following await

Illustration:

code... code... code... await X(); ... code... code... code...

Rearranged:

code... code... code... var x = X(); await X; code... code... code...
^                                  ^          ^                     ^
+---- portion 1 -------------------+          +---- portion 2 ------+

Basically the method executes like this:

  1. It executes everything up to await

  2. It calls the GetSomethingAsync method, which does its thing, and returns something that will complete 2 seconds in the future

    So far we're still inside the original call to button1_Click, happening on the main thread, called from the message loop. If the code leading up to await takes a lot of time, the UI will still freeze. In our example, not so much

  3. What the await keyword, together with some clever compiler magic, does is that it basically something like "Ok, you know what, I'm going to simply return from the button click event handler here. When you (as in, the thing we're waiting for) get around to completing, let me know because I still have some code left to execute".

    Actually it will let the SynchronizationContext class know that it is done, which, depending on the actual synchronization context that is in play right now, will queue up for execution. The context class used in a Windows Forms program will queue it using the queue that the message loop is pumping.

  4. So it returns back to the message loop, which is now free to continue pumping messages, like moving the window, resizing it, or clicking other buttons.

    For the user, the UI is now responsive again, processing other button clicks, resizing and most importantly, redrawing, so it doesn't appear to freeze.

  5. 2 seconds later, the thing we're waiting for completes and what happens now is that it (well, the synchronization context) places a message into the queue that the message loop is looking at, saying "Hey, I got some more code for you to execute", and this code is all the code after the await.

  6. When the message loop gets to that message, it will basically "re-enter" that method where it left off, just after await and continue executing the rest of the method. Note that this code is again called from the message loop so if this code happens to do something lengthy without using async/await properly, it will again block the message loop

There are many moving parts under the hood here so here are some links to more information, I was going to say "should you need it", but this topic is quite broad and it is fairly important to know some of those moving parts. Invariably you're going to understand that async/await is still a leaky concept. Some of the underlying limitations and problems still leak up into the surrounding code, and if they don't, you usually end up having to debug an application that breaks randomly for seemingly no good reason.


OK, so what if GetSomethingAsync spins up a thread that will complete in 2 seconds? Yes, then obviously there is a new thread in play. This thread, however, is not because of the async-ness of this method, it is because the programmer of this method chose a thread to implement asynchronous code. Almost all asynchronous I/O don't use a thread, they use different things. async/await by themselves do not spin up new threads but obviously the "things we wait for" may be implemented using threads.

There are many things in .NET that do not necessarily spin up a thread on their own but are still asynchronous:

  • Web requests (and many other network related things that takes time)
  • Asynchronous file reading and writing
  • and many more, a good sign is if the class/interface in question has methods named SomethingSomethingAsync or BeginSomething and EndSomething and there's an IAsyncResult involved.

Usually these things do not use a thread under the hood.


OK, so you want some of that "broad topic stuff"?

Well, let's ask Try Roslyn about our button click:

Try Roslyn

I'm not going to link in the full generated class here but it's pretty gory stuff.

Formalist
  • 349
  • 3
  • 12
Lasse V. Karlsen
  • 380,855
  • 102
  • 628
  • 825
  • 17
    So it's basically what the OP described as "*Simulating parallel execution by scheduling tasks and switching between them*", isn't it? – Bergi May 24 '16 at 20:11
  • Seems like it, but instead of scheduling and switching back and forth, seeing if it can proceed, it waits until it's flagged and KNOWS that it can proceed. –  May 24 '16 at 20:30
  • 7
    @Bergi Not quite. The execution is truly parallel - the asynchronous I/O task is ongoing, and requires no threads to proceed (this is something that has been used long before Windows came around - MS DOS also used asynchronous I/O, even though it didn't have multi-threading!). Of course, `await` *can* be used the way you describe it as well, but generally isn't. Only the callbacks are scheduled (on the thread pool) - between the callback and the request, no thread is needed. – Luaan May 25 '16 at 08:48
  • @Luaan: Ah, yes, that's the implementation of `GetSomethingAsync`, which I didn't consider. Only the code in the `async` function is scheduled like a task. – Bergi May 25 '16 at 12:30
  • 4
    That's why I wanted to explicitly avoid talking too much about what that method did, as the question was about async/await specifically, which does not create its own threads. Obviously, they can be used to wait *for* threads to complete. – Lasse V. Karlsen May 25 '16 at 14:13
  • 27
    @LasseV.Karlsen -- I'm ingesting your great answer, but I'm still hung up on one detail. I understand that the event handler exists, as in step 4, which allows the message pump to continue pumping, but *when* and *where* does the "thing that takes two seconds" continue to execute if not on a separate thread? If it were to execute on the UI thread, then it would block the message pump anyway while it's executing because it has to execute *some time* on the same thread..[continued]... – rory.ap Dec 08 '16 at 15:39
  • 2
    If "the thing that takes 2 seconds" is one of the operations that Stephen Cleary describes in [his answer](http://stackoverflow.com/a/37419836/2704659) (and [referenced blog post](http://blog.stephencleary.com/2013/11/there-is-no-thread.html), namely an IO operation, then I understand. But is that what your answer assumes? – rory.ap Dec 08 '16 at 15:41
  • It depends entirely on what *kind* of task this is but simplified here's what happens. Some code wakes up in response to the task completing. It may be that this code executes as part of *whatever* that just completed, it may be that it executes as part of an interrupt handling on the processor level. This code posts a message to the message queue of the UI by calling into the synchronization context tied to the task (which is then tied to the message queue), then the code exits. – Lasse V. Karlsen Dec 08 '16 at 15:59
  • The message loop is still pumping messages, and at some point it gets to the message posted by that task completion thingamajig. To process this message, it calls into the task system which then calls the continuation that represents the rest of your task, *after* the call to `await`. And yes, this will indeed "block" the message loop, but if the code is written properly, only for a very short time. – Lasse V. Karlsen Dec 08 '16 at 16:01
  • 1
    In fact, almost every message the message pump is processing is a "blocking" call to some code to process the message, like button clicks, mouse movement, window resizing, etc. The important part is that if this code is written properly it only takes a very short time to execute before returning to the message pump code. We only talk about *blocking the ui* when such code takes a long time. "long time" here is relative, it may be anywhere from 16ms to hours (16ms is about what you need to stay under to get 60fps refresh rate in games/similar). – Lasse V. Karlsen Dec 08 '16 at 16:02
  • @LasseVågsætherKarlsen if you said Web Request don't spin up a new thread, how do you explain about my question https://stackoverflow.com/questions/48366871/async-await-create-new-thread-output-window-show-that ? – KevinBui Mar 18 '18 at 05:22
  • That I don't know enough about the actual implementation of these classes and methods, they don't use threads for their main work, but apparently they do use threads on the threadpool to schedule their continuation or similar. The important part is that this is not due to async/await, but rather just how they decided to implement these things. – Lasse V. Karlsen Mar 18 '18 at 10:21
  • 8
    I like your explanation with the message pump. How does your explanation differs when there is no message pump like in console application or web server? How the reentrace of a method is achieved? – Puchacz Jul 10 '18 at 13:44
  • @LasseVågsætherKarlsen if i understand well, in all cases the main thread (in this case the UI thread) will never execute the 2 seconds Task right ? as if the 2 seconds Task is being delegated to another component ? – Soufien Hajji Feb 21 '19 at 10:53
  • Exactly what the 2 second task does is up to the programmer of that task. It can be executed on the main thread, or a secondary thread, or may not even need to execute at all, it might be an I/O event that is pending. – Lasse V. Karlsen Feb 21 '19 at 14:34
  • Curious what is responsible for notifying or listening...how is that nonblocking? – Switch386 Aug 29 '19 at 19:15
  • 3
    What IMHO is missing in your answer (apart from which it did help me), is to state clearly that all this is futile if GetSomethingAsync() doesn't do something really asynchronous, like starting a thread, doing asynchronous I/O or the like. Everything will be executed in order if not, no responsiveness would be gained. (If I understood all that correctly, that is.) – Eike Oct 29 '19 at 14:20
  • @LasseV.Karlsen [This response](https://stackoverflow.com/a/39196946/1232087) seems to indicate "that there is another thread involved". Third line of that post reads "You have to put the "work" on another Thread". I'm not sure what the writer of that post means to say there because your post and many other I've read seem to indicate the `there is no another thread`. – nam Dec 06 '19 at 00:44
  • That response details something else, and that is how UI's are updated in .NET, WinForms and WPF rely on having a "main thread" that should at all/most times be free to update the UI, which needs code to execute. If you tie up this main thread doing stuff that takes time, the main thread is not free to update the UI and may lead to the UI hanging. In particular, if you want your long-running task to update the UI as part of its job, you very frequently see nothing change until the work is done. – Lasse V. Karlsen Dec 06 '19 at 09:51
  • The way to solve *that* issue is to actually create another thread, but that has nothing to do with async code, and everything to do with how WPF and WinForms handle UI updates. So in that case, yes, you have to *create* that other thread and run the long-running code on it, and leave the main thread free. Again, this has nothing really to do with "async" code. – Lasse V. Karlsen Dec 06 '19 at 09:52
  • If anyone is still interested in the Mads Torgersen: Inside C# Aync episode (which was uploaded in 2010), it is still viewable in (at least) [this capture](https://web.archive.org/web/20151120235256/https://channel9.msdn.com/Shows/Going+Deep/Mads-Torgersen-Inside-C-Async) by the WayBackMachine. It took almost a minute to start streaming though. – Zach Mierzejewski Dec 31 '21 at 18:41
  • Is "await" similar to "yield return" in coroutines in Unity? (not the actions of it but the behaviour it achieves. Ie. no additional thread. leaves the method and comes back. achieves concurrency with a single thread, etc) – whiteSkar May 12 '22 at 19:06
  • 1
    @whiteSkar sort-of, it is similar in the sense that it gets help from the compiler by rewriting the method, and yes, it gets a similar state machine that can resume "mid-way", but the rewrite will involve more stuff related to tasks. – Lasse V. Karlsen May 13 '22 at 06:35
  • I'm confused, please tell me anyone, if I run two asynchronous function Stimulationly, will it run on two threads. I mean in ensure future () method. @LasseV.Karlsen – Siddharth Jun 26 '22 at 15:19
  • `6. When the message loop gets to that message, it will basically "re-enter" that method where it left off` - does the re-enter happen immediately or what is the concept? – variable Jul 06 '22 at 10:12
  • @variable The concept here is that the method has been rewritten to be packaged up as a state machine, with a flag that dictates what portion of it is that is to be executed. The message put into the message loop asks the message pump to call into the state machine, which will then execute the relevant portion of the original method. Basically, there is a method on the state machine object that says "execute the next bit". – Lasse V. Karlsen Jul 11 '22 at 10:16
  • Bravo, well said. – Heytham AlShayeb Apr 25 '23 at 07:22
159

I explain it in full in my blog post There Is No Thread.

In summary, modern I/O systems make heavy use of DMA (Direct Memory Access). There are special, dedicated processors on network cards, video cards, HDD controllers, serial/parallel ports, etc. These processors have direct access to the memory bus, and handle reading/writing completely independently of the CPU. The CPU just needs to notify the device of the location in memory containing the data, and then can do its own thing until the device raises an interrupt notifying the CPU that the read/write is complete.

Once the operation is in flight, there is no work for the CPU to do, and thus no thread.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • Just to make it clear.. I understand the high level of what happens when using async-await. Regarding the no thread creation - there is no thread only in I/O requests to devices which like you said have their own processors which handles the request itself? Can we assume ALL I/O requests are handled on such independent processors meaning use Task.Run ONLY on CPU bound actions? – CodeMonkey Jul 10 '17 at 14:00
  • 1
    @YonatanNir: It's not just about separate processors; any kind of event-driven response is naturally asynchronous. [`Task.Run` is most appropriate for CPU-bound actions](https://blog.stephencleary.com/2013/10/taskrun-etiquette-and-proper-usage.html), but it has a handful of other uses as well. – Stephen Cleary Jul 10 '17 at 14:04
  • 3
    I finished reading your article and there is still something basic I don't understand since I'm not really familiar with the lower level implementation of the OS. I got what you wrote up to where you wrote: "The write operation is now “in flight”. How many threads are processing it? None." . So if there are no threads, then how does the operation itself done if not on a thread? – CodeMonkey Jul 10 '17 at 14:31
  • @YonatanNir: In the example of writing to a disk, the bytes are being written to disk by the disk controller using DMA. But there are other examples where there isn't a separate processor. E.g., timers, or waiting for an HTTP response. – Stephen Cleary Jul 10 '17 at 14:35
  • OK so what happens there where there is no separate processor? How is the action itself done such that there is no need for a thread? – CodeMonkey Jul 10 '17 at 14:43
  • @YonatanNir: It just registers a callback. – Stephen Cleary Jul 10 '17 at 14:44
  • 44
    This is the missing piece in a thousands of explanation!!! There is actually someone doing the work in the background with I/O operations. It is not a thread but another dedicated hardware component doing its job! – the_dark_destructor Nov 10 '17 at 11:29
  • @StephenCleary Where is the state of the state machine in an async method stored before it is resumed? – Prabu Jan 25 '18 at 23:40
  • 2
    @PrabuWeerasinghe: The compiler creates a struct that holds the state and local variables. If an await needs to yield (i.e., return to its caller), that struct is boxed and lives on the heap. – Stephen Cleary Jan 26 '18 at 15:31
  • @StephenCleary So how do you explain my question https://stackoverflow.com/questions/48366871/async-await-create-new-thread-output-window-show-that WebClient async method create a new thread ??? – KevinBui Mar 18 '18 at 05:25
  • 3
    @KevinBui: Asynchronous work depends on the presence of thread pool threads (both worker threads and I/O threads). In particular, I/O Completion Ports require dedicated I/O threads to handle completion requests from the OS. All asynchronous I/O requires this, but the benefit of asynchrony is that you don't need a thread *per request*. – Stephen Cleary Mar 18 '18 at 23:31
  • Hi, one question, How do we know how many I / O threads are running in parallel on the separate I/o processor? And how does this work for HTTP async requests? Is there also another processor in charge of the request threads? – Marcelo Flores Jun 03 '19 at 20:08
  • 1
    @MarceloFlores: There isn't a separate I/O processor. I/O threads are special thread pool threads that only handle completion notifications. HTTP async requests are just like any other I/O; the request is made and when the response comes in, an I/O thread handles that completion notification. There are no request threads. – Stephen Cleary Jun 03 '19 at 21:17
  • @StephenCleary you mentioned _modern I/O systems make heavy use of DMA (Direct Memory Access)_ I have one question though - if memory bus is relinquished to DMA controller then how CPU will perform its operation, isn't it has to sit idle for that time ? (I understand CPU needs memory bus to fetch and execute instructions) – rahulaga-msft Jun 03 '20 at 20:25
  • @rahulaga_dev: The entire point of DMA is that the CPU can operate independently. I'm not familiar enough with modern hardware architectures to speak to specifics. – Stephen Cleary Jun 03 '20 at 21:03
  • 1
    This blog finally made it click. You follow the computation so far down, that threads don't exist :) – wilmol May 13 '21 at 04:34
  • 1
    Isn't this kinda wrong? It's right in the IO case, but what if your async task is just doing computational modelling or something else purely CPU bound? In that case the task is being executed on a separate thread from the pool, right? As developers we must protect data against multi-threaded access because that might be where are async code ends up. – noelicus May 25 '21 at 16:14
  • 5
    @noelicus: The original question was whether `async`/`await` start new threads, and they do not. If you have an `async` modifier on a synchronous method (no `await`), then the compiler will warn you that it will run synchronously (directly on the calling thread). For CPU-bound work, it's common to use `await Task.Run`, in which case the `Task.Run` is what makes it run on a thread pool thread. – Stephen Cleary May 25 '21 at 17:13
  • @StephenCleary you star. `Task.Run`. That's the piece I was missing :D – noelicus May 28 '21 at 16:32
100

the only ways that a computer can appear to be doing more than 1 thing at a time is (1) Actually doing more than 1 thing at a time, (2) simulating it by scheduling tasks and switching between them. So if async-await does neither of those

It's not that await does neither of those. Remember, the purpose of await is not to make synchronous code magically asynchronous. It's to enable using the same techniques we use for writing synchronous code when calling into asynchronous code. Await is about making the code that uses high latency operations look like code that uses low latency operations. Those high latency operations might be on threads, they might be on special purpose hardware, they might be tearing their work up into little pieces and putting it in the message queue for processing by the UI thread later. They're doing something to achieve asynchrony, but they are the ones that are doing it. Await just lets you take advantage of that asynchrony.

Also, I think you are missing a third option. We old people -- kids today with their rap music should get off my lawn, etc -- remember the world of Windows in the early 1990s. There were no multi-CPU machines and no thread schedulers. You wanted to run two Windows apps at the same time, you had to yield. Multitasking was cooperative. The OS tells a process that it gets to run, and if it is ill-behaved, it starves all the other processes from being served. It runs until it yields, and somehow it has to know how to pick up where it left off the next time the OS hands control back to it. Single-threaded asynchronous code is a lot like that, with "await" instead of "yield". Awaiting means "I'm going to remember where I left off here, and let someone else run for a while; call me back when the task I'm waiting on is complete, and I'll pick up where I left off." I think you can see how that makes apps more responsive, just as it did in the Windows 3 days.

calling any method means waiting for the method to complete

There is the key that you are missing. A method can return before its work is complete. That is the essence of asynchrony right there. A method returns, it returns a task that means "this work is in progress; tell me what to do when it is complete". The work of the method is not done, even though it has returned.

Before the await operator, you had to write code that looked like spaghetti threaded through swiss cheese to deal with the fact that we have work to do after completion, but with the return and the completion desynchronized. Await allows you to write code that looks like the return and the completion are synchronized, without them actually being synchronized.

BoltClock
  • 700,868
  • 160
  • 1,392
  • 1,356
Eric Lippert
  • 647,829
  • 179
  • 1,238
  • 2,067
  • Other modern high-level languages support similar explicitly cooperative behavior, too (i.e. function does some stuff, yields [possibly sending some value/object to the caller], continues where it left off when control is handed back [possibly with additional input supplied]). Generators are plenty big in Python, for one thing. – JAB May 24 '16 at 20:49
  • 2
    @JAB: Absolutely. Generators are called "iterator blocks" in C# and use the `yield` keyword. Both `async` methods and iterators in C# are a form of *coroutine*, which is the general term for a function that knows how to suspend its current operation for resumption later. A number of languages have coroutines or coroutine-like control flows these days. – Eric Lippert May 24 '16 at 20:53
  • 2
    The analogy to yield is a good one - it's cooperative multitasking *within one process.* (and thereby avoiding the system stability issues of system-wide cooperative multitasking) – user253751 May 24 '16 at 21:26
  • 3
    I think the concept of "cpu interrupts" being used for IO, is not know about a lot of modem "programmers", hence they think a thread needs to wait for each bit of IO. – Ian Ringrose May 25 '16 at 09:03
  • @EricLippert Async method of WebClient actually creates additional thread, see here https://stackoverflow.com/questions/48366871/async-await-create-new-thread-output-window-show-that – KevinBui Mar 18 '18 at 05:29
  • I think the sentence 'A method can return before its work is complete.' is misleading. The method does not return in the traditional sense of 'reaching the end of the method scope and returning a value'. The method yields execution, essentially saying 'you can put me on hold, I will let you know when I want to continue executing'. I saw someone suggesting in another article that a better name for the 'await' keyword would have been 'yieldUntil' and I wholeheartedly agree, it would have been much more descriptive of what 'await' does – user469104 May 14 '19 at 13:14
  • @user469104: The sentence is misleading when taken out of context, but the context is that it is embedded inside two paragraphs that explain what you just explained again in your comment, so I'm not sure how your critique is actionable. As for the choice of keyword: we considered variations on `yield` and many, many more alternatives; we rejected the `yield` forms as too likely to be confused with `yield return` in the minds of people new to the feature. It's rather late to complain now; the time to make that particular critique was 2011. – Eric Lippert May 14 '19 at 13:54
  • @EricLippert I added my comment as a hint for other people who like me have been thrown off by language of 'await allows the method to return early' when in reality no 'return' occurs (in the traditional sense of a method execution reaching its end of scope) but rather a yield of execution to later be resumed. I personally found your answer to be unclear on this particular point why I added the comment as a hint for future readers who might be in the same boat. Not a criticism of you or the answer as such, just that particular part / wording. – user469104 May 14 '19 at 16:17
  • 1
    @user469104: The entire point of my answer's final paragraphs is to contrast *completion* of a workflow, which is a fact about the state of the workflow, with *return* which is a fact about flow of control. As you note, there is no requirement in general that a workflow be completed before it returns; in C# 2, `yield return` gave us workflows that returned before they completed. `async` workflows are the same; they return before they are complete. – Eric Lippert May 14 '19 at 16:50
  • @user469104 `yielduntil` does seem a lot clearer. And maybe `yielding` instead of `async`. – MarredCheese Sep 18 '21 at 02:28
34

I am really glad someone asked this question, because for the longest time I also believed threads were necessary to concurrency. When I first saw event loops, I thought they were a lie. I thought to myself "there's no way this code can be concurrent if it runs in a single thread". Keep in mind this is after I already had gone through the struggle of understanding the difference between concurrency and parallelism.

After research of my own, I finally found the missing piece: select(). Specifically, IO multiplexing, implemented by various kernels under different names: select(), poll(), epoll(), kqueue(). These are system calls that, while the implementation details differ, allow you to pass in a set of file descriptors to watch. Then you can make another call that blocks until the one of the watched file descriptors changes.

Thus, one can wait on a set of IO events (the main event loop), handle the first event that completes, and then yield control back to the event loop. Rinse and repeat.

How does this work? Well, the short answer is that it's kernel and hardware-level magic. There are many components in a computer besides the CPU, and these components can work in parallel. The kernel can control these devices and communicate directly with them to receive certain signals.

These IO multiplexing system calls are the fundamental building block of single-threaded event loops like node.js or Tornado. When you await a function, you are watching for a certain event (that function's completion), and then yielding control back to the main event loop. When the event you are watching completes, the function (eventually) picks up from where it left off. Functions that allow you to suspend and resume computation like this are called coroutines.

Community
  • 1
  • 1
gardenhead
  • 2,299
  • 2
  • 19
  • 17
33

await and async use Tasks not Threads.

The framework has a pool of threads ready to execute some work in the form of Task objects; submitting a Task to the pool means selecting a free, already existing1, thread to call the task action method.
Creating a Task is matter of creating a new object, far way faster than creating a new thread.

Given a Task is possible to attach a Continuation to it, it is a new Task object to be executed once the thread ends.

Since async/await use Tasks they don't create a new thread.


While interrupt programming technique are used widely in every modern OS, I don't think they are relevant here.
You can have two CPU bonded tasks executing in parallel (interleaved actually) in a single CPU using aysnc/await.
That could not be explained simply with the fact that the OS support queuing IORP.


Last time I checked the compiler transformed async methods into DFA, the work is divided into steps, each one terminating with an await instruction.
The await starts its Task and attach it a continuation to execute the next step.

As a concept example, here is a pseudo-code example.
Things are being simplified for the sake of clarity and because I don't remember all the details exactly.

method:
   instr1                  
   instr2
   await task1
   instr3
   instr4
   await task2
   instr5
   return value

It get transformed into something like this

int state = 0;

Task nextStep()
{
  switch (state)
  {
     case 0:
        instr1;
        instr2;
        state = 1;

        task1.addContinuation(nextStep());
        task1.start();

        return task1;

     case 1:
        instr3;
        instr4;
        state = 2;

        task2.addContinuation(nextStep());
        task2.start();

        return task2;

     case 2:
        instr5;
        state = 0;

        task3 = new Task();
        task3.setResult(value);
        task3.setCompleted();

        return task3;
   }
}

method:
   nextStep();

1 Actually a pool can have its task creation policy.

Margaret Bloom
  • 41,768
  • 5
  • 78
  • 124
  • Upon encountering await the control goes back to the caller. I get this point. But does the thread that called the async function get released into the threadpool? For example in a windows app. – variable Aug 20 '21 at 16:55
  • @variable I'd have to brush up on how exactly it works on .NET but yes. The just called async function returned, meaning that the compiler created an awaiter and appended a continuation to it (that will be called by the Task's awaiter when the awaited event, which is truly asynchronous, finishes). So the thread has nothing more to do and can be returned to the pool, meaning that it can pick up other work. – Margaret Bloom Aug 20 '21 at 17:26
  • I was wondering whether the UI always gets assigned the same thread due to the syncronziarion context, do you know? In that case the thread will not be returned to the pool and will be used by UI thread to run the code following the async method call. I'm a novice in this area. – variable Aug 20 '21 at 17:33
  • @variable [It seems you have to manually call the app dispatcher to make sure the code runs in the UI thread](https://stackoverflow.com/questions/49440017/access-ui-control-on-ui-thread-using-async-a-event). Though that code smells bad to me. [This](https://stackoverflow.com/questions/37971802/async-call-on-gui-event) is a better example. Apparently, the matter is a bit more complex and involves the SynchronizationContext of the GUI thread. ... – Margaret Bloom Aug 20 '21 at 18:49
  • ... If the caller of the `async` function has a SynchronizationContext (like the GUI thread has), the continuation is wrapped in a call that will schedule it in the original context. See [this](https://devblogs.microsoft.com/pfxteam/await-synchronizationcontext-and-console-apps/). Your handler needs to be `async` for this to work. – Margaret Bloom Aug 20 '21 at 18:50
  • `So the thread has nothing more to do and can be returned to the pool`. Can you help me here: you say thread has nothing more to do, I know that you are right. But can you help me understand because I'm thinking like this: the thread does have more work - for example it has to continue running the code of the caller once control is returned to caller. Is my thinking wrong? – variable Aug 20 '21 at 19:04
  • @variable A worker thread is executing a loop that picks up an element of work (a function to call) and executes it (call it). It then repeats. Like in an event loop. I don't remember exactly how this is implemented in C# but the idea should be siimilar. – Margaret Bloom Aug 20 '21 at 20:01
22

Here is how I view all this, it may not be super technically accurate but it helps me, at least :).

There are basically two types of processing (computation) that happen on a machine:

  • processing that happen on the CPU
  • processing that happen on other processors (GPU, network card, etc.), let's call them IO.

So, when we write a piece of source code, after compilation, depending on the object we use (and this is very important), processing will be CPU bound, or IO bound, and in fact, it can be bound to a combination of both.

Some examples:

  • if I use the Write method of the FileStream object (which is a Stream), processing will be say, 1% CPU bound, and 99% IO bound.
  • if I use the Write method of the NetworkStream object (which is a Stream), processing will be say, 1% CPU bound, and 99% IO bound.
  • if I use the Write method of the Memorystream object (which is a Stream), processing will be 100% CPU bound.

So, as you see, from an object-oriented programmer point-of-view, although I'm always accessing a Stream object, what happens beneath may depend heavily on the ultimate type of the object.

Now, to optimize things, it's sometimes useful to be able to run code in parallel (note I don't use the word asynchronous) if it's possible and/or necessary.

Some examples:

  • In a desktop app, I want to print a document, but I don't want to wait for it.
  • My web server servers many clients at the same time, each one getting his pages in parallel (not serialized).

Before async / await, we essentially had two solutions to this:

  • Threads. It was relatively easy to use, with Thread and ThreadPool classes. Threads are CPU bound only.
  • The "old" Begin/End/AsyncCallback asynchronous programming model. It's just a model, it doesn't tell you if you'll be CPU or IO bound. If you take a look at the Socket or FileStream classes, it's IO bound, which is cool, but we rarely use it.

The async / await is only a common programming model, based on the Task concept. It's a bit easier to use than threads or thread pools for CPU bound tasks, and much easier to use than the old Begin/End model. Undercovers, however, it's "just" a super sophisticated feature-full wrapper on both.

So, the real win is mostly on IO Bound tasks, task that don't use the CPU, but async/await is still only a programming model, it doesn't help you to determine how/where processing will happen in the end.

It means it's not because a class has a method "DoSomethingAsync" returning a Task object that you can presume it will be CPU bound (which means it maybe quite useless, especially if it doesn't have a cancellation token parameter), or IO Bound (which means it's probably a must), or a combination of both (since the model is quite viral, bonding and potential benefits can be, in the end, super mixed and not so obvious).

So, coming back to my examples, doing my Write operations using async/await on MemoryStream will stay CPU bound (I will probably not benefit from it), although I will surely benefit from it with files and network streams.

Simon Mourier
  • 132,049
  • 21
  • 248
  • 298
  • 1
    This is quite a good answer using the theadpool for cpu bound work is poor in sense that TP threads should be used to offload IO operations. CPU bound work imo should be blocking with caveats of course and nothing precludes the use of multiple threads. – davidcarr Dec 11 '18 at 14:31
20

I'm not going to compete with Eric Lippert or Lasse V. Karlsen, and others, I just would like to draw attention to another facet of this question, that I think was not explicitly mentioned.

Using await on it's own does not make your app magically responsive. If whatever you do in the method you are awaiting on from the UI thread blocks, it will still block your UI the same way as non-awaitable version would.

You have to write your awaitable method specifically so it either spawn a new thread or use a something like a completion port (which will return execution in the current thread and call something else for continuation whenever completion port gets signaled). But this part is well explained in other answers.

Andrew Savinykh
  • 25,351
  • 17
  • 103
  • 158
11

I try to explain it bottom up. Maybe someone find it helpful. I was there, done that, reinvented it, when made simple games in DOS in Pascal (good old times...)

So... Every event driven application has an event loop inside that's something like this:

while (getMessage(out message)) // pseudo-code
{
   dispatchMessage(message); // pseudo-code
}

Frameworks usually hide this detail from you but it's there. The getMessage function reads the next event from the event queue or waits until an event happens: mouse move, keydown, keyup, click, etc. And then dispatchMessage dispatches the event to the appropriate event handler. Then waits for the next event and so on until a quit event comes that exits the loop and finishes the application.

Event handlers should run fast so the event loop can poll for more events and the UI remains responsive. What happens if a button click triggers an expensive operation like this?

void expensiveOperation()
{
    for (int i = 0; i < 1000; i++)
    {
        Thread.Sleep(10);
    }
}

Well the UI becomes nonresponsive until the 10 second operation finishes as the control stays within the function. To solve this problem you need to break up the task into small parts that can execute quickly. This means you cannot handle the whole thing in a single event. You must do a small part of the work, then post another event to the event queue to ask for continuation.

So you would change this to:

void expensiveOperation()
{
    doIteration(0);
}

void doIteration(int i)
{
    if (i >= 1000) return;
    Thread.Sleep(10); // Do a piece of work.
    postFunctionCallMessage(() => {doIteration(i + 1);}); // Pseudo code. 
}

In this case only the first iteration runs then it posts a message to the event queue to run the next iteration and returns. It our example postFunctionCallMessage pseudo function puts a "call this function" event to the queue, so the event dispatcher will call it when it reaches it. This allows all other GUI events to be processed while continuously running pieces of a long running work as well.

As long as this long running task is running, its continuation event is always in the event queue. So you basically invented your own task scheduler. Where the continuation events in the queue are "processes" that are running. Actually this what operating systems do, except that the sending of the continuation events and returning to the scheduler loop is done via the CPU's timer interrupt where the OS registered the context switching code, so you don't need to care about it. But here you are writing your own scheduler so you do need to care about it - so far.

So we can run long running tasks in a single thread parallel with the GUI by breaking up them into small chunks and sending continuation events. This is the general idea of the Task class. It represents a piece of work and when you call .ContinueWith on it, you define what function to call as the next piece when the current piece finishes (and its return value is passed to the continuation). But doing all this chaining splitting up work into small pieces manually is a cumbersome work and totally messes up the layout of the logic, because the entire background task code basically a .ContinueWith mess. So this is where the compiler helps you. It does all this chaining and continuation for you under the hood. When you say await you tell the compiler that "stop here, add the rest of the function as a continuation task". The compiler takes care of the rest, so you don't have to.

While this task piece chaining doesn't involve creating threads and when the pieces are small they can be scheduled on the main thread's event loop, in practice there is a worker thread pool that runs the Tasks. This allows better utilization of CPU cores and also allows the developer to run a manually written long Task (which would block a worker thread instead of the main thread).

Calmarius
  • 18,570
  • 18
  • 110
  • 157
  • What a perfect exemplfying I admire your explanation +1. All old guys should explain similar concepts in the way you have already done here given that as a person from Z generation I don’t know what was happened and how it happened in the past. – Soner from The Ottoman Empire Feb 02 '21 at 05:56
  • I finally got that. Everyone says "There is no thread", but nobody says somehow that there is one, namely one (at least one) from the thread pool. Those are also threads or am I getting something wrong? – deralbert Jun 16 '21 at 21:41
  • 1
    @deralbert The thread pools are there because Tasks are not only used to implement async-await. You can create a Task object manually that does an expensive operation without chunking. When you run that, it would block a worker thread from pool instead of the main thread. But still the small chunks of async-await task pieces execute quickly, they wouldn't block, therefore they can even be run on the main thread without extra threads. (Updated the answer to be less misleading.) – Calmarius Jul 07 '21 at 09:56
4

Summarizing other answers:

Async/await is generally created for IO bound tasks as by using them, the calling thread doesn't need to be blocked. This is especially useful in case of UI threads as we can ensure that they remain responsive while a background operation is being performed (like fetching data to be displayed from a remote server)

Async doesn't create it's own thread. The thread of the calling method is used to execute the async method till it finds an awaitable. The same thread then continues to execute the rest of the calling method beyond the async method call. Note that within the called async method, after returning from the awaitable, the reminder of the method could be executed using a thread from the thread pool - the only place a separate thread comes into picture.

vaibhav kumar
  • 885
  • 1
  • 11
  • 13
  • 1
    Good summary, but I think it should answer 2 more questions in order to give the full picture: 1. What thread the awaited code is executed on? 2. Who controls/configures the mentioned thread pool - the developer or the runtime environment? – stojke Dec 23 '18 at 14:05
  • 1. In this case, mostly the awaited code is a IO bound operation which wouldn't use CPU threads. If it is desired to use await for CPU bound operation, a separate Task could be spawned. 2. The thread in the thread pool is managed by the the Task scheduler which is part of the TPL framework. – vaibhav kumar Dec 28 '18 at 03:20
3

This does not directly answer the question, but I think it provides some interesting additional information:

Async and await does not create new threads by itself. BUT depending on where you use async-await, the synchronous part BEFORE the await may run on a different thread than the synchronous part AFTER the await (for example ASP.NET and ASP.NET core behave differently).

In UI-Thread based applications (WinForms, WPF) you will be on the same thread before and after. But when you use async-await on a Thread pool thread, the thread before and after the await may not be the same.

A great video on this topic

mfluehr
  • 2,832
  • 2
  • 23
  • 31
Welcor
  • 2,431
  • 21
  • 32
1

Actually, async await chains are state machine generated by CLR compiler.

async await however does use threads that TPL are using thread pool to execute tasks.

The reason the application is not blocked is that the state machine can decides which co-routine to execute, repeat, check, and decides again.

Further reading:

What does async & await generate?

Async Await and the Generated StateMachine

Asynchronous C# and F# (III.): How does it work? - Tomas Petricek

Edit:

Okay. It seems like my elaboration is incorrect. However I do have to point out that state machines are important assets for async awaits. Even if you take-in asynchronous I/O you still need a helper to check if the operation is complete therefore we still need a state machine and determine which routine can be executed asychronously together.

Steve Fan
  • 3,019
  • 3
  • 19
  • 29