6

We are writing unit tests for async code using MSTest and Moq.

So we have some code that looks something like :

var moq = new Mock<Foo>();
moq.Setup(m => m.GetAsync())
   .Returns(Task.FromResult(10));

Or like this on projects that have a more recent version of Moq

var moq = new Mock<Foo>();
moq.Setup(m => m.GetAsync())
   .ReturnsAsync(10);

Looking at the Moq implementation of ReturnsAsync :

public static IReturnsResult<TMock> ReturnsAsync<TMock, TResult>(this IReturns<TMock, Task<TResult>> mock, TResult value) where TMock : class
{
  TaskCompletionSource<TResult> completionSource = new TaskCompletionSource<TResult>();
  completionSource.SetResult(value);
  return mock.Returns(completionSource.Task);
}

Both methods seem to be the same under the hood. Both create a TaskCompletionSource, call SetResult and return the Task

So far so good.

But short running async methods are optimized to act synchronously. This seems to imply that TaskCompletionSource is always synchronous, which would also seem to suggest that context handling and any related issues that can occur would never happen.

So if we had some code that was doing some async no-no's, like mixing awaits, Wait() and Result, that these problems would not be detected in unit testing.

Would there be any advantage to creating an extension method which always yields control? Something like :

public async Task<T> ReturnsYieldingAsync<T>(T result)
{
    await Task.Yield();
    return result;
}

In this case we would have a method that is guaranteed to execute asynchronously.

The perceived advantage would be detecting bad asynchronous code. For example, it could catch any deadlocking or exception swallowing during unit testing.

I am not 100% sure this is the case, so I would really be interested in hearing what the community has to say.

swestner
  • 1,881
  • 15
  • 19
  • I'd say go for `ReturnsYieldingAsync`, but even more so, to experience potential deadlocks you'd need to install a synchronization context in your test runners, something like [`AsyncPump`](http://blogs.msdn.com/b/pfxteam/archive/2012/01/20/10259049.aspx). – noseratio Jan 15 '16 at 11:42
  • I think you should try writing a test case that proves 1. it will work as expected and 2. you can create a test case that follows the test case from 1 exactly but never calls `ReturnsYieldingAsync` and fails. – David Pine Jan 19 '16 at 12:58
  • 1
    @Noseratio Nice find on the `AsyncPump`. The major issue I think we are running into is that the test running into is that the synchronization context of the test runner is null. Swapping out the `AsyncPump` seems to be the way to go as shown here http://stackoverflow.com/questions/14087257/how-to-add-synchronization-context-to-async-test-method – swestner Jan 19 '16 at 16:31
  • @DavidPine I agree, and we have been trying to create a test case to that would hit a deadlock. The above code is a bit naive. It will actually await as soon as it enters the chain. What we ended up having to do is chaining a task through the Moq `Callback` method, and then calling the `Return` on the task used in the callback. Even then we are not hitting any deadlocks. As the above comments suggest, this is probably and issue with `AsyncPump`, and I am now testing different variations of it out. I will update the question or add an answer when I have it all sorted out. – swestner Jan 19 '16 at 16:41

1 Answers1

3

When I first started talking about testing asynchronous code four years ago (!), I would encourage developers to test along the async axis for their mocks. That is, test along the result axis (success, fail) as well as the async axis (sync, async).

Over time, though, I've loosened up on this. When it comes to testing my own code, I really only test the synchronous success/fail paths, unless there's a clear reason to also test the asynchronous success path for that code. I don't bother with asynchronous failure testing at all anymore.

Stephen Cleary
  • 437,863
  • 77
  • 675
  • 810
  • In our case we have come across a number of instances of mixed async patterns. In large part this is a training issue, but it is also something we can't overlook. Your Nito library helped us reproduce the deadlocks consistently. Thanks! Ultimatley though we are probably going to tests these types of issues through E2E tests. – swestner Feb 02 '16 at 18:27
  • I do have one outstanding curiosity that was core to the original question. Do you know if `TaskCompletionSource` always run synchronously? It was an assumption of the original question which would be nice to know. – swestner Feb 02 '16 at 18:30
  • @swestner: If you call `SetResult` (or similar), then the task is completed before the method call returns. There may still be continuations of that task that haven't completed, though. – Stephen Cleary Feb 03 '16 at 00:15