Unit Testing and Isolation

Question

I'm aware that unit tests should be completely independent of one another, meaning that the order of operations should never be important.

How do you handle a scenario where you have test that loads a large document over the network, yet needs other tests to verify information on said document?

I do not want the document requested for each test that has to check this or that.

It works fine if the document is stored as a static member of the test class, and the initial test that fetches it runs first. Then the subsequent tests can all do their own thing with the cached copy.

But, again, order of operations with unit tests is wrong, as they should be independent.

What is the best practice in this case? I'd prefer to have them all as separate tests and not rolled into one single, big test that can assert all sorts of things about the document after it's been fetched.

My current scenario is Visual Studio 2015 Update 3 and C#, but I guess it can be applied to any similar setup.

In Visual Studio's test runner, you can set up "playlists" that will let you run them in an order, but it still feels wrong.

Edit: I'm leaning towards the static approach, in which case each test has to get the document via a single method. I guess a singleton-type approach (if null, go and get it, otherwise return it).

But any feedback is good.

You can put one-time setup logic in a method marked with `[TestInitialize]` from `Microsoft.VisualStudio.TestTools.UnitTesting` — Brian, Jul 20 '16 at 19:44
Why do you need to get this resource from the remote location? That makes the tests dependent on whatever is running there. — Sign, Jul 20 '16 at 19:46
@ScottChamberlain Oops, I was looking for `[ClassInitialize]`. — Brian, Jul 20 '16 at 19:46
@Patrick See http://stackoverflow.com/questions/1873191/why-testinitialize-gets-fired-for-every-test-in-my-visual-studio-unit-tests — Brian, Jul 20 '16 at 19:46
@Sign I have mocks in place but certain tests require verifying the actual document itself. — Patrick, Jul 20 '16 at 19:46
@Brian `[ClassInitialize]` is what I needed, thanks. I was going to say submit it as an answer and I'd accept, but it looks like Thaoden beat you to it. — Patrick, Jul 20 '16 at 19:52
`[ClassInitialize]` is run once per `[TestClass]`, there is the even more general `[AssemblyInitialize]` that gets run once per test dll. — Scott Chamberlain, Jul 20 '16 at 19:54
@ScottChamberlain Thanks Scott, these tests that need the real document are isolated to a single class, but it's good information. `[ClassInitialize]` resolves the main issue here. — Patrick, Jul 20 '16 at 19:57

Scott Chamberlain · Accepted Answer · 2016-07-20T19:53:18.380

The best option is don't fetch the document over the network, bundle it as part of the test payload.

That being said, if you need one time logic that happens at startup you can add a [AssemblyInitialize] to a test that will run once when the test assembly is loaded. You can do your test configuration there.

[TestClass]
public class GlobalTestData
{
    private static byte[] _document;

    public static byte[] Document
    {
        get
        {
            return (byte[])_document.Clone();
        }
    }

    [AssemblyInitialize()]
    public static void DownloadDocument(TestContext context)
    {
        _document = DownloadDocument();
    }

    private static byte[] DownloadDocument()
    {
        //...
    }
}

Just throw that above class in to your project then any test can use GlobalTestData.Document and will get a fresh unaltered copy of the byte[].

score 1 · Answer 2 · answered Jul 20 '16 at 19:50

Unit Tests test things in isolation. If you fetch your document over a network connection, it's not isolated. Try to mock the serving of the document.

You could also fetch the document once in a [TestInitialize] method as suggested by Brian in the comments or in a [ClassInitialize] static method and store it in a local variable, the difference being that TestInitialize runs before every test and ClassInitialize only before the first test is run.

UPDATE:

If you need the proper document for your tests, you're looking at integration tests. In my opinion, integration tests are perfectly fine to be dependent on one another as they check for correct behaviour if multiple systems have to work together. Just make sure they're not in the same class as your unit tests, as these should be fast and independent to run.

Thanks, I'm looking at both `ClassInitialize` and `AssemblyInitialize` to see what is the best idea here. The problem is this document is created by a third-party, and I need to verify things such as structure, size, expected data, etc. The structure is not under my control, so the mocks can pass but the code will fail. — Patrick, Jul 20 '16 at 20:01

Unit Testing and Isolation

2 Answers2