Multi threading unit test

Question

I have a C# method that will be called multiple times using different threads. So I want to create a unit test that will test this method on several threads, but I'm not sure I'm doing it right.

This is my unit test without threading:

    [TestMethod]
    public void FromLocalPoints()
    {
        var projectedCoordinates = this.ConvertFromLocalPoints();
        foreach (var projectedCoordinate in projectedCoordinates)
        {
            Debug.Write(projectedCoordinate.X);
            Debug.Write("; ");
            Debug.WriteLine(projectedCoordinate.Y);
        }
    }

this.ConvertFromLocalPoints() is calling the actual method I want to test.

I've created a delegate, an event and an handler:

public delegate void ReprojectCompleteHandler(IEnumerable<Coordinate> projectedCoordinates);
public event ReprojectCompleteHandler ReprojectCompleteEvent;
private void ReprojectHandler(IEnumerable<Coordinate> projectedCoordinates)
{
        Debug.WriteLine("Conversion is complete");
}

In my TestSetup I listen to my event:

    [TestInitialize]
    public void TestSetup()
    {
        this.ReprojectCompleteEvent += this.ReprojectHandler;
    }

My unit test is:

    [TestMethod]
    public void FromLocalPointsThreaded()
    {
        // Call FromLocalPoints multiple times in separate threads to check if it is thread safe
        for (var i = 0; i < 10; i++)
        {
            var myThread = new Thread(this.ConvertFromLocalPointsThreaded);    
        }

        Debug.WriteLine("FromLocalPointsThreaded is done");
    }

    private void ConvertFromLocalPointsThreaded()
    {
        var projectedCoordinates = this.ConvertFromLocalPoints();

        // Send result to delegate/event:
        if (this.ReprojectCompleteEvent != null)
        {
            this.ReprojectCompleteEvent(projectedCoordinates);
        }
    }

When I run this unit test I get once 'FromLocalPointsThreaded is done' in my output but no 'Conversion is complete'.

What am I missing to get this working? Or should I use a different approach?

Update We're currently switching libraries which is doing the actual conversion. The old library isn't thread-safe so we added locks. The new library should be thread-safe so I want to remove the locks. But I need a unit test which will prove our method using the new lib is really thread-safe.

There is nothing that tells the main thread to wait for the spawned threads. It just exits before they do any work. Look at Thread.Join. — mrmcgreg, Jun 02 '15 at 09:16
In addition to the answers above (both mrmcgreg's and Micky Duncan's), if you wish to measure the performance of your code in a parallelized environment, you might wish to start your thread functions in one run. See [my answer on a different question](http://stackoverflow.com/a/30568340/2144232) on hpw it can be done in C#. — mg30rg, Jun 02 '15 at 09:24

Euphoric · Answer 1 · 2015-06-02T09:38:25.450

One of the properties of good Unit test is that it need to be repeatable. Every time it is run, it should run in same way as before. This is simply not possible with threading. For example, the test might run properly 999 times, but hit the deadlock 1 time. Meaning the thread is not only useless, but gives you false sense of confidence that your code actually doesn't have deadlock.

For testing threading safety, there are few other ways to accomplish it:

Mock out the threading

Extract away the threaded code and replace it with abstraction that can be mocked in test. This way, the unit test code will simulate multiple threads without threading itself being an issue. But this requires you knowing all possible paths threads can go through your code, making it useless for anything complex.

This can also be helped by using immutable data structures and pure methods. The tested code then is limited to small portion of code that will provide synchronization between the threads.

Endurance testing

Design the test to run for extremely long period of time, spawning new threads and calling code all the time. If it runs for few hours without a deadlock, then you get good confidence that there is no deadlock. This cannot be run as part of normal test suite, nor does it give you 100% confidence. But it is much more practical than trying to enumerate all possible ways threads can interact.

+1. My guess about the downvotes is because the OP _might_ know the downsides of threads in unit tests already and he asked specifically about how to test _threaded_ - that said, your explanation of *why it's bad* deserves the up. — Johann Gerell, Jun 02 '15 at 09:15
Thanks for the suggestions. I think the Endurance Testing option will work for us since all I want is the prove my code is thread-safe (I've updated my question). But to prove it I need to show the results in my output, that's why I created the handler. That part doesn't seems to be working. — Paul Meems, Jun 02 '15 at 09:31
@PaulMeems You cannot prove your code is thread-safe until you use some kind of formal method. Which would require completely different approach than you are doing right now. — Euphoric, Jun 02 '15 at 09:37

score 2 · Accepted Answer · answered Jun 02 '15 at 11:29

As suggested I switched my approach. I'm now using:

    [TestMethod]
    public void FromLocalPointsParallel()
    {
        var loop = new int[10];
        Parallel.ForEach(
            loop,
            item =>
                {
                    var projectedCoordinates = this.ConvertToLocalPoints();
                    foreach (var projectedCoordinate in projectedCoordinates)
                    {
                        var x = projectedCoordinate.X;
                        var y = projectedCoordinate.Y;
                        Assert.IsFalse(double.IsInfinity(x) || double.IsNaN(x), "Projected X is not a valid number");
                        Assert.IsFalse(double.IsInfinity(y) || double.IsNaN(y), "Projected Y is not a valid number");
                    }
                });
    }

This unit test is doing what I need: call my conversion method in different threads. And it already proved its use as I detected a dictionary I used wasn't thread-safe. I fixed it and now I'm confident the new library is thread-safe.

Thanks all for your help and suggestions.

Multi threading unit test

2 Answers2