Is it possible to unit test exceptions in Commands wrapping an async lambda?

Question

Consider the following highly simplified viewmodel for fetching and showing a list of projects:

public class ProjectListViewModel
{
    private readonly IWebService _webService;
    public ICommand RefreshCommand { get; }
    // INotifyPropertyChanged implementation skipped for brevity
    public ObservableCollection<Project> Projects { get; set; }

    public ProjectListViewModel(IWebService serverApi)
    {
        _serverApi = serverApi;
        // ICommand implemented by Xamarin.Forms
        RefreshCommand = new Command(async () => await RefreshAsync());
    }

    private async Task RefreshAsync()
    {
        try
        {
            Projects = await _webService.GetProjectsAsync();
        }
        catch (TaskCanceledException)
        {
            // Empty (task only cancelled when we are navigating away from page)
        }
    }
}

Using NUnit and Moq, I'm trying test that when GetProjectsAsync throws a TaskCanceledException, the ViewModel will catch it. The closest I get is this:

[Test]
public void When_Refreshing_Catches_TaskCanceledException()
{
    // Arrange
    webService = new Mock<IServerApi>();
    webService.Setup(mock => mock.GetProjectsAsync())
        .ThrowsAsync(new TaskCanceledException());
    vm = new ProjectListViewModel(webService.Object);

    // Act and assert
    Assert.That(() => vm.RefreshCommand.Execute(null), Throws.Nothing);
}

The test passes, but unfortunately it's faulty - it still passes if I throw e.g. Exception instead of TaskCanceledException. As far as I know, the reason is that the exception doesn't bubble up past the command lambda, async () => await RefreshAsync(), so no exception thrown by GetProjectsAsync will ever be detected by the test. (When running the actual app however, the TaskCanceledException will bubble up and crash the app if not caught. I suspect this is related to synchronization contexts, of which I have very limited understanding.)

It works if I debug the test - if I mock it to throw Exception, it will break on the line with the command/lambda definition, and if I throw TaskCanceledException, the test will pass.

Note that the results are the same if I use Throws instead of ThrowsAsync. And in case it's relevant, I'm using the test runner in ReSharper 2016.2.

Using nUnit, is it possible at all to unit test exceptions thrown when executing "async" commands like this? Is it possible without writing a custom Command implementation?

Have you looked to this? http://stackoverflow.com/a/15636814/4625433 — Amaury Levé, Nov 01 '16 at 13:46

score 6 · Accepted Answer · answered Nov 01 '16 at 13:48

6

Your problem is here:

new Command(async () => await RefreshAsync())

This async lambda is converted to an async void method by the compiler.

In my article on async best practices, I explain why the exception cannot be caught like this. async methods cannot propagate their exceptions directly (since their stack can be gone by the time the exception happens). async Task methods solve this naturally by placing the exception on their returned task. async void methods are unnatural, and they have nowhere to place the exception, so they raise it directly on the SynchronizationContext that was current at the time the method started.

In your application, this is the UI context, so it's just like it was thrown directly in an event handler. In your unit test, there is no context, so it's thrown on a thread pool thread. I think NUnit's behavior in this situation is to catch the exception and dump it to the console.

Personally, I prefer using my own asynchronous-compatible ICommand such as AsyncCommand in my Mvvm.Async library (also see my article on asynchronous MVVM commands):

new AsyncCommand(_ => RefreshAsync())

which can then be naturally unit tested:

await vm.RefreshCommand.ExecuteAsync(null); // does not throw

Alternatively, you can provide your own synchronization context in the unit test (using, e.g., my AsyncContext):

// Arrange
webService = new Mock<IServerApi>();
webService.Setup(mock => mock.GetProjectsAsync())
    .ThrowsAsync(new TaskCanceledException());
vm = new ProjectListViewModel(webService.Object);

// Act/Assert
AsyncContext.Run(() => vm.RefreshCommand.Execute(null));

In this case, if there was an exception, Run would propagate it.

answered Nov 01 '16 at 13:48

Stephen Cleary

437,863
77
675
810

Thanks! I'll have a look at your library. – cmeeren Nov 01 '16 at 14:07
1

`...is converted to an async void method by the compiler.` Is this because the signature of `Command` is expecting an `Action`? – Kenneth K. Nov 01 '16 at 14:08
@StephenCleary, in your `AsyncCommand`, is there any way to trigger `OnCanExecuteChanged`? Otherwise, I assume the UI won't check for changes to `CanExecute`? The Xamarin.Forms implementation of Command has a method called `ChangeCanExecute` which must be manually called when `CanExecute` needs to be re-evaluated by the UI. – cmeeren Nov 01 '16 at 14:39
@cmeeren: The default behavior of `AsyncCommand` is to return `false` for `CanExecute` when the command is executing. If you want custom `CanExecute` logic, then you can pass a `CanExecute` implementation to the `AsyncCommand` constructor and call `OnCanExecuteChanged` when the return value may have changed. – Stephen Cleary Nov 01 '16 at 15:16
@StephenCleary Thanks. Since Mvvm.Async is not on Nuget, and since I only need it to make a unit test pass, I'd rather not use it in production code (though I did test it, and it worked). I ended up installing AsyncEx in my test project (targeting .NET 4.5, so couldn't install just AsyncEx.Context) and using `Assert.That(() => AsyncContext.Run(() => vm.RefreshCommand.Execute(null)), Throws.Nothing);`, which works perfectly. – cmeeren Nov 02 '16 at 07:33
@cmeeren: Of course [it's on NuGet](https://www.nuget.org/packages/Nito.Mvvm.Async/1.0.0-eta-02). :) – Stephen Cleary Nov 02 '16 at 11:07
Oh sorry, I didn't check the "Include prerelease" checkbox. Thanks, I'll look more carefully at it then. Is the included `NotifyTask` your "canonical" version of the NotifyTaskCompletion that you described in one of your articles? Because there seems to be something similar in AsyncEx. – cmeeren Nov 02 '16 at 12:16
@cmeeren: Yes, `NotifyTask` is the modern version of `NotifyTaskCompletion`. I recommend targeting 4.6 or higher; Microsoft dropped 4.5 support almost a year ago (though they do support 4.5.2 still). – Stephen Cleary Nov 02 '16 at 12:47

score 0 · Answer 2 · answered Nov 01 '16 at 13:54

Since async void (which is what the handler to your command is) is basically "fire and forget" and you can't await for it in the test I would suggest unit testing the RefreshAsync() method (you may want to make it internal or public), this can be easily done in NUnit:

if you are asserting exceptions being thrown:

[Test]
public async Task Test_Exception_RefreshAsync(){
        try
        {
            await vm.RefreshAsync();
            Assert.Fail("No exception was thrown");
        }
        catch (NotImplementedException e)
        {
            // Pass
        }
}

or simply

[Test]
public async Task Test_RefreshAsync(){
    var vm = new ProjectListViewModel(...);
    await vm.RefreshAsync();
    //Assertions here
}

or as other answer state you can create your own AsyncCommand that you can await on.

Thanks for the suggestion, but I'd rather not unit test private methods. In general, they are implementation details and apt to change. — cmeeren, Nov 01 '16 at 14:06
Right, it's a (controversial) trade-off, upside being that it's simpler than using a new 3rd party library. — brakeroo, Nov 01 '16 at 14:51

Is it possible to unit test exceptions in Commands wrapping an async lambda?

2 Answers2