Is there an attribute to avoid cascade unit test failures in MSTest?

Question

I am attempting to follow more closely the practice of 'one assertion per unit test'. It is not always possible, but I do find it useful for pinpointing errors.

For example I might have something like:

var toTest;

[TestInitialize]
Init() {
  toTest = // Create toTest
}

[TestMethod]
TestIsCreated() {
  Assert.IsNotNull(toTest);
}

[TestMethod]
TestIsProperty1Setup() {
  Assert.IsNotNull(toTest.Property1);
  // Maybe some more tests on Property1
}

// More tests for other properties...

The problem is if the creation of toTest were to fail, then all of the other unit tests fail too. So we can add some check for this like so:

...

[TestMethod]
TestIsProperty1Setup() {
  if (toTest == null) Assert.Inconclusive()
  Assert.IsNotNull(toTest.Property1);
  // Maybe some more tests on Property1
}

...

This will stop cascade failures and points to the exact problem. If the // Create toTest line returns a null then I get exactly one unit test failing, and a bunch more inconclusive tests. When I fix the one failing test then everything else passes.

There are two places where this falls down. One is the fact that I now have repeating code at the start of almost all unit tests.

The second problem is when I want to setup and check a more complex object (in this dummy example toTest also has an array):

[TestMethod]
TestArrayIsNotNull() {
  if (toTest == null) Assert.Inconclusive();
  Assert.IsNotNull(toTest.Array);
}

[TestMethod]
TestArrayIsInitilizedWithOneElement() {
  if (toTest == null) Assert.Inconclusive();
  if (toTest.Array == null) Assert.Inconclusive();
  Assert.AreEqual(1, toTest.Array.Count())
}

[TestMethod]
TestArrayIsInitilizedWithCorrectElement() {
  if (toTest == null) Assert.Inconclusive();
  if (toTest.Array == null) Assert.Inconclusive();
  if (toTest.Array.Count() != 1) Assert.Inconclusive();
  // Assert something about toTest.Array[0]
}

Now there is more duplicate code for each individual test, and more importantly the Asserts have to be kept in sync between different tests meaning that one small change can ripple across many unit tests.

Ideally I would like to have an attribute that sits on the top of each test and only runs the test if 'prerequisite' tests pass. It might look something like this:

[TestMethod]
[OnlyRunIfOtherTestPasses(TestIsCreated())]
TestArrayIsNotNull() {
  Assert.IsNotNull(toTest.Array);
}

[TestMethod]
[OnlyRunIfOtherTestPasses(TestArrayIsNotNull())]
TestArrayIsInitilizedWithOneElement() {
  Assert.AreEqual(1, toTest.Array.Count())
}

[TestMethod]
[OnlyRunIfOtherTestPasses(TestArrayIsInitilizedWithOneElement())]
TestArrayIsInitilizedWithCorrectElement() {
  // Assert something about toTest.Array[0]
}

The unit tests are now very simple - there is no duplication and keeping tests synchronised with their 'prerequisites' is now automatic.

Is there something like this? Or is there an entirely different feature that I can use to get the same effect?

I am aware that there are almost certainly issues with referring to a test method in an attribute to another test method - you would probably have to use a string which is hard coded to the method name (so what happens if the method name changes, or the method doesn't exist). I am sure there are also potential problems with circular references. But despite all these issues, someone must have addressed this issue. Can anyone point it out to me?

IMHO you have too much logic in your unit tests. They should be concise and not contain a lot of logic. They also shouldn't depend on other tests. I think you need to re-think your approach. — Belogix, Aug 15 '13 at 11:05
When you say 'rethink your approach' do you mean rethink how the unit tests are written (I shouldn't check if toTest is null all the time), or to rethink the dependencies of the class under test (toTest shouldn't have an array that must be set up)? — Jonny, Aug 15 '13 at 11:13
Duplicate of: http://stackoverflow.com/questions/15871933/how-to-stop-mstest-tests-execution-on-first-failure — BartoszKP, Aug 15 '13 at 11:21

BartoszKP · Answer 1 · 2013-08-15T11:16:27.310

0

Dependency between tests is discouraged. Possible solution to your problem: if you throw an exception if the creation of "toTest" should fail (or use an ordinary Assert) in the test initialization method, then I believe tests from the group won't be run at all. You could also use static class initialization ( http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting.classinitializeattribute.aspx ). However, all tests will still be marked as failed.

edited Aug 15 '13 at 11:16

answered Aug 15 '13 at 11:07

BartoszKP

34,786
15
102
130

If I throw an exception in the TestInitialize method then all unit tests fail. I am aiming for exactly one unit test to fail if the creation does not work so that it is very easy to pinpoint the problem. – Jonny Aug 15 '13 at 11:15

Is there an attribute to avoid cascade unit test failures in MSTest?

1 Answers1