18

I have 2 questions in regard doing integration testing using VS 2010

First, I'm really in need of finding a way to execute these testing methods in the order I want them to. Note: I know in Unit Testing, methods should run standalone from anything else, but these are integration tests which I do depend on the order of which method runs first.

On the same note, is there a way to keep a local variable through running the tests? For example like the following code which right now fails.

[TestClass]
public class UnitTest1
{
    int i = 0;
    [TestMethod]
    public void TestMethod1()
    {
        i = 5;
    }

    [TestMethod]
    public void TestMethod2()
    {
        Assert.AreEqual(5, i);
    }
}

So is there a way to do any of these?

Ray
  • 12,101
  • 27
  • 95
  • 137

2 Answers2

21

To execute tests in a specific order I followed the next steps:

In a test project with test1, test2 and test3

1 Right click on the project 'Add'->'new test..."
2 Select 'Ordered Test'
3 Double click in the file that appears "OrderedTest1.orderedtest"

alt text

4 Build the project if was not build previously
5 From the list of available test select the test you want and order them

alt text

From that point on there appears a new test in the test list editor

alt text

It is an extra test that runs the enclosed tests in the correct order, but if you run all the test in the project carelessly the tests included in the ordered list will be executed twice so you need to somehow manage lists or test categories to avoid that.
I tried disabling the individual tests but that also disables the ordered test, I don't know a better way to do so.

Cristian T
  • 2,245
  • 3
  • 25
  • 45
  • Thanks Cristian for your time on the pix. I marked it as answer. Though, I have to say this may be too much for what I was looking for, so I'll probably have to include multiple tests into one single test method for now. I also contacted Rob, he said different test runners behave differently, VS seems to execute these test methods in random order since it treats them as Unit Tests which should be able to run at any order. Wish they have an attribute to mark the test class as [IntegrationTest] then execute the methods in the order as they appear. – Ray Jan 14 '11 at 19:01
  • @Cristian, after reading so many 'It can't be done' and 'you should rename your tests to order them alphabetically' etc your answer is the first one that actually works without being a hack. Thank you for this! – callisto May 27 '14 at 07:54
6

It is best practice to use functions to set up the tests and to clean them up, by using the attributes [TestInitialize] and [TestCleanUp] or [ClassInitialize] and [ClassCleanup].
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.unittesting(v=VS.100).aspx

The next code is an example of a similar thing to what you want:

[TestClass]
public class UnitTest1
{
    int i=0;

    [TestInitialize]
    public void Setup()
    {
        i = 5;
    }

    [TestMethod]
    public void TestMethod1()
    {
        Assert.AreEqual(5, i);
    }
}

The function SetUp will be called before executing each test. If you need to pass the value from one test to the other you might want to consider using a static variable which is not recommended due to the indeterministic order of execution.

Usually there is a way to avoid needing a specific order by using the setup/cleanup technique, but it is true that this might not be true for very complex integration tests.
If there is no possible way to avoid having them to reorder you can consider merging them in one, breaking again the best practice of having only one assert per test, but if they are so much dependent one from the other it might be even better this way, as in this case one test failing might compromise the result of the others.

EDIT: May be using ordered tests answers question 1, and using static variables question 2: http://msdn.microsoft.com/en-us/library/ms182631.aspx

Cristian T
  • 2,245
  • 3
  • 25
  • 45
  • Thank you Cristian for your input. I'm aware of the attributes you mentioned and unfortunately these are not what I am after. I'm running integration tests against a test data source so even if they fail in the middle somewhere I can restore the data back. I saw a Rob Conery video on SpecFlow in which he seemed to have these achieved http://tekpub.com/view/concepts/5 I just don't know how he did it. – Ray Jan 13 '11 at 20:58
  • I just added edited the question and added the idea of the ordered tests. Let me know if that really works. – Cristian T Jan 13 '11 at 21:15
  • Thanks Cristian, I just checked out ordered tests as you pointed out, but I don't know how exactly to get them to execute? I selected the tests I want to the right hand panel, but when I hit run test, it run all the tests I have in the entire solution, any ideas? – Ray Jan 13 '11 at 21:22
  • I elaborated a new answer for the ordered test topic as I though it was worth experimenting a bit with them myself – Cristian T Jan 14 '11 at 07:38