Is it incorrect to call async methods synchronously in unit tests?

Question

I'm new to the async/await world and I'm trying to figure out the benefits of writing asynchronous unit tests for async methods. That is, must a unit test for an async method call that async method asynchronously? If it calls the async method synchronously using Task.Run(), what is lost? In the latter case, code coverage isn't impacted as far as I can see.

The reason I'm asking this is because our mocking software (we use TypeMock) can't support async/await. (They say there is a legitimate reason for this lack of support, and I don't disagree with them.) By calling the async methods synchronously in the unit tests, we can workaround this issue. However, I'd like to know whether we are cutting any corner by doing this.

For example, let's say I have the following async method:

public async Task<string> GetContentAsync(string source)
{
    string result = "";
    // perform magical async IO bound work here to populate result
    return result;
}

The following is the ideal unit test which doesn't work:

[TestMethod()]
public async Task GetContentAsyncTest()
{
    string expected = "thisworks";
    var worker = new Worker();
    // ...mocking code here that doesn't work!
    string actual = await worker.GetContentAsync();
    Assert.AreEqual(expected, actual);
}

But this works, and it does provide the code coverage we need. Is this OK?

[TestMethod()]
public void GetContentAsyncTest()
{
    string expected = "thisworks";
    var worker = new Worker();
    // mocking code here that works!
    string actual = Task.Run(() => worker.GetContentAsync()).Result;
    Assert.AreEqual(expected, actual);
}

I'm not really a fan of this question; the title is a bit broad. In most cases, you aren't testing whether or not something took X long or Y long, you're just saying "If it's done, make sure this happened" or "Make sure this happened if it failed". Whether it's Async or not doesn't come into play in the vast majority of cases. — George Stocker, May 29 '15 at 21:50
Async processing is mainly to free up the the thread pool or IIS worker processes — fuzzybear, May 29 '15 at 21:52
@George Stocker I know, but the fact that I see async unit test examples out there makes me think there is a reason why people write them. What do you suggest for the title then? Can't you give me a chance to improve it before downvoting? A bit trigger happy? — Zoomzoom, May 29 '15 at 21:52
@saj, I know what async/await's purpose is. But this is in the context of writing unit tests. — Zoomzoom, May 29 '15 at 21:52
I think the question is fine +1. But its a pretty broad question.... Most async things won't care, some might... — AK_, May 29 '15 at 22:00
Ok, I think I see why you think it's broad. Asking "what are the benefits" is pretty open ended (though I didn't really expect it to be). Any suggestion? — Zoomzoom, May 29 '15 at 22:01
Try this https://msdn.microsoft.com/en-us/magazine/dn818494.aspx, talks about the differences — fuzzybear, May 29 '15 at 22:02
async..await is just syntactic sugar for Tasks, so in that regard the second option should be just fine. — GolezTrol, May 29 '15 at 22:04
Remember that TypeMock does some things quite differently from other mocking frameworks. It's really quite invasive sometimes, and actually does some things to your code _after it is built_. Not all mocking frameworks will have such issues. — John Saunders, May 29 '15 at 22:09
@GolezTrol It seems to be more than syntactic sugar as it was enough to break TypeMock. But your comment does help bolster my confidence. — Zoomzoom, May 29 '15 at 22:09
I'm using nunit and I've async/await in unit tests everywhere... — Ryan Chu, May 29 '15 at 22:11
@RyanChu Thanks for sharing. Do you guys have a specific reason behind that? Did that implementation help you discover any issues specific to async/await? — Zoomzoom, May 29 '15 at 22:13
@Zoomzoom hmmm... Not really. async is like poison, we just make everything up the stack async as well :p — Ryan Chu, May 29 '15 at 22:37

score 4 · Accepted Answer · answered May 29 '15 at 23:12

must a unit test for an async method call that async method asynchronously?

No, but it is most natural to do so.

If it calls the async method synchronously using Task.Run(), what is lost?

Nothing really. It's slightly less performant, but to a degree that you will probably never notice.

You'd probably want to use GetAwaiter().GetResult() instead of Result to avoid AggregateException wrappers in your failure tests. And you can also just call the method directly; no need to wrap it in a Task.Run.

They say there is a legitimate reason for this lack of support, and I don't disagree with them.

Oh, I certainly disagree with them. :)

Does that mean they can't unit test iterator blocks, either? The exact same reasoning would apply...

The only more serious problem with not supporting async unit tests is if the code under test assumed its context would handle synchronization. This is common, for example, in moderately complex View Models.

In that case, you'd need to install a context in which to execute the async code (e.g., my AsyncContext type) unless you're using a unit test framework that automatically provides one (as of this writing, only xUnit does AFAIK).

To your second point regarding the loss of performance - even if it was significant, it still wouldn't be a concern at all for unit tests. — Zoomzoom, May 30 '15 at 00:56
@Zoomzoom: I disagree. To a certain degree, [performance of unit tests is important](http://xunitpatterns.com/Slow%20Tests.html). — Stephen Cleary, May 30 '15 at 03:41
Stephen - I see. Hard to argue with that! For my situation though, test performance was nowhere near as important as code coverage. — Zoomzoom, May 30 '15 at 05:48
@Zoomzoom Be careful putting too much weight on code coverage. Sometimes it's nowhere near as important as one might think http://stackoverflow.com/questions/90002/what-is-a-reasonable-code-coverage-for-unit-tests-and-why — craftworkgames, May 30 '15 at 13:01

score 2 · Answer 2 · answered May 29 '15 at 22:49

2

If you use xUnit instead of MSTest, your ideal solution (async test) will work.

answered May 29 '15 at 22:49

Richard Szalay

83,269
19
178
237

I'm using xUnit, but I am getting weird deadlocks when testing async methods. I even tried ConfigureAwait(false). – Vin Shahrdar Aug 22 '18 at 18:49

Is it incorrect to call async methods synchronously in unit tests?

2 Answers2