3

It is known that synchronous waiting on an async method leads to deadlocks (see, for example Don't Block on Async Code)

I have the following code in an event handler for a button-click in a Windows Forms application (i.e. the code is invoked with a UI SynchronizationContext installed).

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://www.google.com"));
Task<HttpResponseMessage> t = client.SendAsync(request);
t.Wait();
var response = t.Result;

I fully expected the code to deadlock on clicking the button. However, what I actually see is synchronous waiting - the dialog becomes unresponsive for a while, and then accepts events as usual. I consistently see deadlocks when I try to synchronously wait on client async methods. However, synchronously waiting on library async methods like SendAsync or ReadAsByteArrayAsync seems not to deadlock. Can someone explain this behaviour?

Don't implementations of async methods in .NET libraries use await statements internally, so that the continuations have to be marshalled back to the original SynchronizationContext?

Note: If I define a client method, say

public async Task<byte[]> wrapperMethod()
{
    var client = new HttpClient();
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://www.google.com"));
    var response = await client.SendAsync(request);
    return await response.Content.ReadAsByteArrayAsync();
}

and then say byte[] byteArray = wrapperMethod().Result; in the button click handler, I do obtain a deadlock.

svick
  • 236,525
  • 50
  • 385
  • 514
Anirudh
  • 55
  • 1
  • 5
  • 1
    As the accepted answer describes well, there is no deadlock because most Microsoft .NET code does not resume on a captured context. However, be careful depending on this. In particular, I believe `HttpClient` does use `await` (and - incorrectly - resume on a captured context) on some mobile platforms. – Stephen Cleary Dec 26 '15 at 14:45

2 Answers2

4

Don't implementations of async methods in .NET libraries use await statements internally?

Generally, no. I have yet to see a single implementation in the .NET framework that uses async-await internally. It does use tasks and continuations but not the compiler magic the async and await keywords bring.

Using async-await is simple as the code looks synchronous but actually runs asynchronously. But that simplicity has a very small price in performance.

For most consumers this prices is worth paying, but the framework itself tries to be as performant as possible.

However, synchronously waiting on library async methods like SendAsync or ReadAsByteArrayAsync seems not to deadlock.

The deadlock is a result of the default behaviour of await. When you await an uncompleted task the SynchronizationContext is captured and when it's completed the continuation is resumed on that SynchronizationContext (if it exists). When there's no async, await, captured SynchronizationContext, etc. this kind of deadlock can't happen.

HttpClient.SendAsync specifically uses TaskCompletionSource to return a task without marking the method as async. You can see that in the implementation on github here.

Most task-returning methods added to existing classes for async-await simply build a task using the already existing asynchronous API (i.e. BeginXXX/EndXXX). For example this is TcpClient.ConnectAsync:

public Task ConnectAsync(IPAddress address, int port)
{
    return Task.Factory.FromAsync(BeginConnect, EndConnect, address, port, null);
}

When you do use async-await though you avoid the deadlock by using ConfigureAwait(false) when you don't need to capture the SynchronizationContext. It's recommended that libraries should alway use it unless the context is needed (e.g. a UI library).

i3arnon
  • 113,022
  • 33
  • 324
  • 344
  • 2
    I think the "performance" part is quite misleading in the context of using `async/await` when creating new libraries. Does the BCL use `async/await`? Possibly not. Should *you* use `async/await` when creating new libraries? Definitely. Here's a quote from Microsoft's "TPL Performance Improvements in .NET 4.5": *"Support for the new “await” keyword is built on top of Task continuations, but the await support is highly optimized and generally much more efficient than an “off the shelf” continuation"* – Kirill Shlenskiy Dec 26 '15 at 10:59
  • Full paper which talks about `Task`'s single-continuation optimisation introduced in .NET 4.5 and implications for `async/await` can be found here: http://blogs.msdn.com/b/pfxteam/archive/2011/11/10/10235962.aspx – Kirill Shlenskiy Dec 26 '15 at 11:01
  • Clarification: I'm trying to understand why synchronous waiting on library methods does not seem to deadlock, rather than how to avoid such deadlocks. Any ideas on the first question - why does synchronous waiting on the library methods indicated above not deadlock? – Anirudh Dec 26 '15 at 11:04
  • @Anirudh The deadlock happens because awaiting a task resumes on the captured `SynchronizationContext` (unless specified otherwise). In .NET methods there is no use of await to begin with. There's no option to deadlock. There's no async, await, captured SC, etc... – i3arnon Dec 26 '15 at 11:06
  • @KirillShlenskiy Should **you** use async-await when creating new libraries? probably. Does the .NET team always use async-await when creating new libraries? definitely not. – i3arnon Dec 26 '15 at 11:10
  • @KirillShlenskiy though with the addition of `ValueTask` (more on it [on my blog](http://blog.i3arnon.com/2015/11/30/valuetask/)) they would probably be using it more and more as the overhead can be smaller for the right API. – i3arnon Dec 26 '15 at 11:11
  • @i3arnon Thanks for the link to the source. If you happen to know that the BeginXXX/EndXXX methods do not interact with `SynchronizationContext`, I'd be very grateful for some references. Task.ContinueWith does rely on the current SyncContext, for example, see http://stackoverflow.com/questions/23350631/what-synchronizationcontext-does-task-continuewith-use. In the meanwhile, I'll try and study the source to try and understand why it does not deadlock. – Anirudh Dec 26 '15 at 11:14
  • @Anirudh no, it doesn't (as the answer you linked to suggests). – i3arnon Dec 26 '15 at 11:20
  • @i3arnon The accepted answer to that question is somewhat misleading. Please see the second answer - it links to [source](http://referencesource.microsoft.com/#mscorlib/system/threading/Tasks/Task.cs,e71924f994d1bb54) saying that `TaskScheduler.Current` is used to schedule the continuation – Anirudh Dec 26 '15 at 11:28
  • @ 1. I disagree that it is misleading (the second answer is). 2. `TaskScheduler` is completely different than `SynchronizationContext`. You can create a wrapper of one around the other with `TaskScheduler.FromCurrentSynchronizationContext` but **you** need to do it, it doesn't happen on its own. – i3arnon Dec 26 '15 at 11:32
  • @Anirudh also, if you [search for usages of that method on the .NET core repository you'll find only tests](https://github.com/dotnet/corefx/search?utf8=%E2%9C%93&q=FromCurrentSynchronizationContext&type=Code). – i3arnon Dec 26 '15 at 11:33
  • 1
    @Anirudh, you're actually onto something when you blame `SynchronizationContext` rather than `async/await` as ultimately it's the `SynchronizationContext` postback which leads to the deadlock, regardless of the higher-level technology used. I would say, however, that *not* capturing the context as part of async `HttpClient` calls is *common sense*. It's only `Task`s that *must* interact with the thread they're started on, that will deadlock out of the box in your repro, and you won't find very many of those in the .NET base class library (I can only think of async `Dispatcher` calls). – Kirill Shlenskiy Dec 26 '15 at 11:39
  • @KirillShlenskiy The usage of the `SynchronizationContext` happens inside `TaskAwaiter`. Specifically in `Task.SetContinuationForAwait`. That's how it's related to async-await. If you don't await a task that code isn't called. Are there any **other** explicit usages of `SynchronizationContext` in the entire .NET framework? maybe, though probably not a lot. – i3arnon Dec 26 '15 at 11:49
  • This discussion clarified a few things for me. Thanks! @i3arnon, if you substantiate "When there's no async, await, captured SynchronizationContext, etc. this kind of deadlock can't happen." from your answer with references showing that the lower level APIs (beginXXX/endXXX/continueWith) do not capture context, I shall be happy to accept your post as the answer. – Anirudh Dec 26 '15 at 11:58
  • @Anirudh That's the wrong way to think about it. You can see who **does capture** the SC in await, which is the `TaskAwaiter`. `BeingXXX/EndXXX` is a pattern. Everyone can write their own methods and some may capture the SC (though there's no real reason to). – i3arnon Dec 26 '15 at 12:01
  • 1
    @i3arnon, thanks for the `ValueTask` info by the way - that's something pretty exciting that I learnt today. – Kirill Shlenskiy Dec 26 '15 at 12:02
  • I think Stream.CopyToAsync to uses await because loops are so hard to build without await. And WebClient captures the SC as an example for that. Horrible API design. No built-in way to tun that feature off. – usr Dec 26 '15 at 15:19
  • @usr [`Stream.CopyToAsyncInternal`](http://referencesource.microsoft.com/#mscorlib/system/io/stream.cs,162) – i3arnon Dec 26 '15 at 16:49
2

You won't cause a deadlock by blocking on most out-of-the box Task-returning .NET calls because they wouldn't internally touch the SynchronizationContext that the Task was started on unless absolutely necessary (two reasons: performance and avoiding deadlocks).

What this means is that even if standard .NET calls did use async/await under the covers (i3arnon said they don't - I won't argue as I simply don't know), they would, beyond any doubt, use ConfigureAwait(false) unless capturing the context is definitely required.

But that's the .NET framework. As for your own code, you will observe a deadlock if you call wrapperMethod().Wait() (or Result) in your client (provided that you're running with a non-null SynchronizationContext.Current - if you're using Windows Forms, this will definitely be the case). Why? Because you're flaunting async/await best practices by not using ConfigureAwait(false) on your awaitables inside async methods that do not interact with the UI, causing the state machine to generate continuations that unnecessarily execute on the original SynchronizationContext.

Kirill Shlenskiy
  • 9,367
  • 27
  • 39
  • They actually use `async/await` quite a bit, e.g.: http://imgur.com/L82cNcr (http://referencesource.microsoft.com/#mscorlib/system/threading/Tasks/Task.cs,9ca6b2f012ce7587) – noseratio Dec 27 '15 at 00:03