How to detect UnobservedTaskException errors in nunit test suites

Question

My team makes extensive use of NUnit unit tests in our C# project. Recently we have started using the Task Parallel Library (TPL) in .NET 4, which has introduced a wrinkle for us.

In the TPL, if a Task is faulted (that is, an exception was thrown while executing the task), that task's Exception property must be retrieved at least once. If it is not, when the Task object is finalized by the garbage collector an exception will be thrown which terminates the process.

It is possible to detect and prevent this by registering a handler for the TaskScheduler.UnobservedTaskException. We have done that for a few of our test cases to reproduce unobserved task exception bugs, but I would rather have some way to modify the way NUnit runs the tests so that for each test, an UnobservedTaskException handler is registered, then after that test a garbage collection is forced to flush out any tasks with unobserved exceptions. I would then like this to cause the test to fail.

How are other teams solving this problem? How are you detecting test cases that pass (that is, complete without any exceptions) but leave one or more Task objects with unobserved exceptions?

JLScott · Accepted Answer · 2011-08-06T02:49:23.000

Here is how I do it:

[Test]
public void ObservesTaskException()
{
    bool wasUnobservedException = false;

    TaskScheduler.UnobservedTaskException += 
        (s, args) => wasUnobservedException = true;

    CauseATaskToThrowInTheSystemUnderTest();

    GC.Collect();
    GC.WaitForPendingFinalizers();

    Assert.That(wasUnobservedException, Is.False);
}

The call to CauseATaskToThrowInTheSystemUnderTest() is a placeholder for whatever you need to do. I recommend wrapping your code in a function like this because it aids in making sure the Task object that throws the exception is unreachable when the GC runs. If the Task object is reachable, the finalizer won't run and this test will not test anything.

Obviously, it is also important to make sure the task has completed before garbage collection occurs. How you do this (if you even can) is dependent on your particular code. Perhaps the flow of your program ensures this is the case. If not, and if your test can access to the Task object, you can do the following:

var continuation = GetTheTaskFromTheSystemUnderTest();
    .ContinueWith(t => {});

CauseATaskToThrowInTheSystemUnderTest();

bool isTaskCompleted = continuation.Wait(SomeSuitableTimeout);

You should add isTaskCompleted to your assertion.

The timeout is there just in case the code is broken in the future--you don't want your test to hang. The value should be very small. If you find you actually need to wait very long for your task, your test suite will likely be too slow for frequent use.

Having access to the tasks you create (including continuation tasks you create on other tasks) is one of the considerations to keep in mind while designing for testability. When doing so, the usual tradeoffs apply. I try to include tests like this when my code creates tasks--this is an important behavior of my code and it needs to be tested.

I eventually arrived at this same answer on my own. Works great. There is a some possibility that the task will be finalized before the call to GC.Collect and GC.WaitForPendingFinalizers, but in practice I've found the probability to be so low as to be of no concern. — anelson, Aug 06 '11 at 04:10
Are you sure this works? I just tried it with both `Task.Run(Throw)` and `Task.Factory.StartNew(Throw)` on a `int Throw() { throw new Exception(); }`, tried a `using()` scope on the task, tried continuation task and tried not having a task variable at all. The event is never raised to me, even when trying to Thread.Sleep() or using ManualResetEvent to ensure the task is completed and give the system some time to finalize the task. — angularsen, Feb 02 '16 at 12:23
Duplicate question: http://stackoverflow.com/questions/21266137/test-for-unobserved-exceptions — angularsen, Feb 02 '16 at 12:30

GregC · Answer 2 · 2011-04-20T19:29:42.927

0

I would keep unit tests deterministic, then do functional testing with CHESS or Jinx.

This would mean that I have no code that relies on concurrent flows in the unit testing logic.

As for testing multiple exceptions packaging/unpackaging, i would test it as if it was a loop: no exceptions, one exception, and two exceptions prove the whole thing.

edited Apr 20 '11 at 19:29

answered Apr 20 '11 at 19:20

GregC

7,737
2
53
67

Fair enough; how would you solve this problem of unobserved exceptions in CHESS or Jinx? – anelson Apr 21 '11 at 23:22
It seems you're testing core functionality of TPL. That is, you're checking out if it still assembles unhandled exceptions in a bunch and passes them to initiating thread correctly. You appear to have a good idea what exceptions are thrown on each thread. I'd unit test the logic on thread, and rely on the fact that dotNET still works when it comes to threads. – GregC Apr 22 '11 at 03:53
Assert that a piece of code throws a certain exception with Assert.Throws(). CHESS and Jinx can run your code in various interleavings and see if your unit tests still hold water, ensuring that shared state is properly protected. – GregC Apr 22 '11 at 03:55
To that end, it's possible to write threaded software without shared state. With care. – GregC Apr 22 '11 at 03:56

How to detect UnobservedTaskException errors in nunit test suites

2 Answers2