How grunty or expensive is C# async/await?

Question

I want to connect 50 USB serial port devices to a single computer and communicate asynchronously (using polling) with each of them.

AFAIK, using C# async/await does not cause the creation of additional threads. However, even though only a single thread is used, the rest of the program maintains responsiveness for non-CPU bound tasks.

For each USB serial port async/await method call, I understand that a new state machine is created by the compiler/CLR.

How many USB serial port methods can be async/awaited before a single thread becomes overloaded? I'm guessing that dynamically creating state machines is CPU heavy. Is the CLR smart enough to offload some of the workload / state machine overhead to other threads or CPU cores?

Example issue: Async/await code development takes place on a new fast computer, but the customer has an old slow computer. Initiating many state machines may work well during development, but perhaps stall upon deployment.

Edit: The following dummy code creates 500 async/awaits, thus causing the creation of 500 state-machines. Compiling and running the code from the command line shows no evidence of increased CPU usage (according to Windows Resource monitor plots).


MyClass[] myClass = new MyClass[numInstances];

for (int i = 0; i < numInstances; i++)
{
    myClass[i] = new MyClass();
    _ = myClass[i].MyMethod(i);
}

Console.ReadLine();

public class MyClass
{
    public async Task MyMethod(int i)
    {
        Console.WriteLine("Opening port" + i);
        await Task.Delay(3000);  // Do USB serial port poll
        Console.WriteLine("Done" + i);
    }
}

There are practical limits, but its not as simple as what you are asking for, so we can't give you a direct response. You will get better feedback if you post an actual scenario that you want feedback on how or if it should be optimised. In general use nested async/await as much as you can, the compiler is smart enough to get the balance right. — Chris Schaller, Aug 03 '22 at 22:55
Initiating state machines is likely to see _MORE_ benefit on slower production environments, or rather responsiveness will become more of an issue on slower resources as each individual task is going to take longer to execute, so you will more likely experience delays if you do not manage your async processing correctly. — Chris Schaller, Aug 03 '22 at 23:14
Think of using async/await as giving the compiler the _option_ to use a State Machine implementation to optimize the execution of the code, but it doesn't force it to do so. — Chris Schaller, Aug 03 '22 at 23:14
Using standard async/await will generally keep you out of trouble, but when you start to involve looping with constructs like [`Task.WhenAll()`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.task.whenall?view=net-6.0) and nesting those looping and parallel processing tasks, then you might consider flattening the process if you experience any issues. — Chris Schaller, Aug 03 '22 at 23:15
Chris, thanks. That is helpful. In your third comment, are you meaning that state-machine code is NOT necessarily generated by the compiler? e.g. it is not deterministic in terms of state-machine creation, from the programmer's point of view? I had not read that previously. — user284898, Aug 03 '22 at 23:39
Chris, I note that you didn't mention the compiler/CLR using additional threads or CPU cores. Is this something that *absolutely* will not happen? — user284898, Aug 03 '22 at 23:44
I note that @StephenCleary posted the following on [another page](https://stackoverflow.com/questions/37419572/if-async-await-doesnt-create-any-additional-threads-then-how-does-it-make-appl): "... For CPU-bound work, it's common to use await Task.Run, in which case the Task.Run is what makes it run on a thread pool thread." — user284898, Aug 03 '22 at 23:51
async/await will _always_ generate a state machine, the compiler does this. What is not deterministic until execution is if this will be executed asynchronously or on a different thread. This is probably the simplest answer I've seen: https://stackoverflow.com/a/28944441/1690217 — Chris Schaller, Aug 03 '22 at 23:54
If you deliberately want one task to execute on a different thread, and to start immediately, then `Task.Run()` is a common solution, but for the love of all things good, do not overuse this, only when you have an explicit CPU heavy process, expecially if you are NOT awaiting the result. If you are not going to await, then inside the task logic you will need to explicitly trap and manage exceptions. Most `Task.Run()` examples I see tend to use _Fire and Forget_, without properly handling the exceptions. — Chris Schaller, Aug 03 '22 at 23:58
For explicit parallel processing of high CPU loads, have a read over https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/task-parallel-library-tpl among other improvements, async/await is designed to simplify code needed to properly un-block the UI thread, it is not fundamentally a performance feature. TPL is specifically designed to manage parallel processing of large lists of tasks where you want specific thread management and throttling. If you are concerned about runtime performance, this might be the solution. — Chris Schaller, Aug 04 '22 at 00:04
I did say to **_"Think"_** of using async/await as giving the compiler the option to use a State Machine... I know that it does, but noobs often associate State Machine with some intensive overhead, when the reality is that the overheads are often significantly smaller than the code you would otherwise write to try and avoid a State Machine implementation or to _roll your own_. State Machine can be kinder to low resourced systems than `Task.Run()` that deliberately executes on a separate threadpool thread. It allows for _Interleaved_ processing as well as thread based processing — Chris Schaller, Aug 04 '22 at 00:14
I have voted to re-open by the way as the change to focus specifically about serial device communications does give us a specific scenario to respond to. If you update the post to include some code that shows how you connect and where async/await is used you might get even more reopen votes — Chris Schaller, Aug 04 '22 at 00:22
In terms of polling a serial connection, if your polling logic is synchronous, then async/await is perfect, especially if the processing of the response is not very intensive. You might `Task.Run()` the processing of a significant result, given that polling implies many cycles will actually do _nothing_. If you wanted to put each USB device on their own thread and poll them in parallel, but this can lead to contention issues that will drop performance. In my IoT projects I often get better throughput if I poll the bus _synchronously_, not in parallel. (so use async/await) — Chris Schaller, Aug 04 '22 at 00:37
Chris, thanks so much, those answers are gold ! I'm unable to post actual serial port code as I am away, but have posted something similar without the serial port code. — user284898, Aug 04 '22 at 00:42
Thanks for the code, in that situation, async/await is perfect. That is not an issue where _State Machine_ becomes a problem. Show an example of simple conditional logic that you might execute, it's what happens inside the task that will determine if threads need to be involved, and TBH the .net runtime will do a pretty good job at this on it's own, await Task.Delay() is probably the worst kind of demo to use, what happens in that block is the significant peice of the puzzle. — Chris Schaller, Aug 04 '22 at 00:45

score 2 · Answer 1 · answered Aug 04 '22 at 02:57

I want to connect 50 USB serial port devices to a single computer and communicate asynchronously (using polling) with each of them.

Serial ports are one of those awkward APIs in .NET because they just don't get the love and attention that other APIs do. For one thing, there's a restriction on naming (they must start with COM) that .NET insists on even though it's not necessary (and many devices - including utilities such as com0com - don't have names starting with COM). For another, even though practically all APIs have been updated to have ReadAsync / WriteAsync methods, SerialPort has not. SerialPort does have a BaseStream property, but I'm not sure if it actually supports asynchronous operations, or if they're always synchronous.

TL;DR: Be sure to do some testing before you assume things will work. With SerialPort, always verify with manual testing.

For each USB serial port async/await method call, I understand that a new state machine is created by the compiler/CLR.

State machines are created as needed for each async method, yes.

I'm guessing that dynamically creating state machines is CPU heavy.

It's heavier on memory. But what's the alternative: creating a thread for each connection? Allocating a state machine (usually a few dozen or maybe hundred bytes) is going to be far more efficient than creating a thread, both in terms of CPU and memory.

Is the CLR smart enough to offload some of the workload / state machine overhead to other threads or CPU cores?

No; they're always created on the current thread. But the offloading would cause more CPU overhead than it would save. Creating the state machine is literally just an allocate and then a shallow-copy of all the local variables.

How many USB serial port methods can be async/awaited before a single thread becomes overloaded?

More generally: how many async methods can be in progress? And the answer is "a lot". Waaaay more than a few hundred; I'd guess probably around 50 million or so even on 10-15-year-old machines, though I haven't tested it. To put it another way: I've never seen anyone run into this limit; some other scalability limit is always hit first.

score 0 · Answer 2 · answered Aug 04 '22 at 08:09

As Stephen said, a state machine is more an indication of where your code stopped so it can pick back up where it left to do the rest of the work. In your case I would worry more about the synchronization context.

When the code encounters an await, it will also record the current synchronization context so it can schedule the continuation on that. There are 3 big ones that you have to take care of. There is a default one where every continuation will take a free thread out of the thread pool and run it's continuation on. There is also one for when you are running in a form or in an ASP.NET framework page. These ones limit all the work to a single thread to run the continuations on. New continuations that are scheduled will wait until the thread is free with it's work.

Assume that after the await, you have some very intensive work that keeps the CPU busy for a while. In the case of the default synchronization context, you will see that all the continuations are working in parallel (multithreaded). If you have more continuations than there are threads in the thread pool, then those will need to wait.

If the synchronization context only allows a single thread, then the next only run if the previous continuation is done (really done, or it has hit another await that isn't completed yet). You can avoid this by using the .ConfigureAwait(false) by telling that you want to use the default synchronization context. Alert: this only works in combination with await because the synchronization context is stored in the state machine, calling _ = myClass[i].MyMethod(i).ConfigureAwait(false); will have no effect as there is no state machine created at this point to save this context in. And this method doesn't change the context of the state machine that it has received.

How grunty or expensive is C# async/await?

2 Answers2