36

Given this test fixture:

[TestClass]
public class MSTestThreads
{
    [TestMethod]
    public void Test1()
    {
        Trace.WriteLine(Thread.CurrentThread.ManagedThreadId);
    }

    [TestMethod]
    public void Test2()
    {
        Trace.WriteLine(Thread.CurrentThread.ManagedThreadId);
    }
}

Running the test with MSTest through Visual Studio or command line prints two different thread numbers (yet they are run sequentially anyway).

Is there a way to force MSTest to run them using a single thread?

BartoszKP
  • 34,786
  • 15
  • 102
  • 130
Paul Stovell
  • 32,377
  • 16
  • 80
  • 108
  • 3
    Automated UI tests that need to run on the same application UI thread, since they all have to use a single static WPF Application class that can't be instantiated more than once (WPF's restriction, not mine). The technique works in NUnit. – Paul Stovell Feb 18 '11 at 04:10
  • 1
    I solved this by creating a new appdomain for each test, executing the body of the test inside that appdomain, and creating a new App instance in each test. I can see why this would be painful in most cases, but in my case, I wrote my own app class, so I wasn't really duplicating any testing by adding that boilerplate. – Merlyn Morgan-Graham Mar 29 '11 at 02:50
  • Also, if you're sharing an App instance, you theoretically could create a UI thread for it, and share that too. – Merlyn Morgan-Graham Mar 29 '11 at 02:53
  • 3
    @Merlyn - can you add an answer about your appdomain-per-test solution - it might be useful to other people – codeulike Mar 06 '12 at 11:48
  • 1
    I was originally trying to get it to run on one thread, but what I really wanted was sequential execution of certain data access framework tests (because of database setup and tear-down dependencies). This can be done with the Ordered Test feature - see this post: http://stackoverflow.com/questions/1544827/is-there-a-way-to-run-unit-tests-sequentially-with-mstests – komma8.komma1 Aug 29 '12 at 19:08
  • @PaulStovell MS Test is not designed for automated UI tests, don't blame the tool if doesn't support something it wasn't designed for. Also, don't try to share something what is not meant to be shared, the Application instance in this case. Create and dispose the Application instance in the test method not in setup. You can still extract shared code into a method and call it from test method. – user3285954 Oct 08 '15 at 19:20
  • VSTest.Console.Exe that suppose to replace MStest uses single thread. See [How does the Visual Studio 2012 test runner apply threading?](http://stackoverflow.com/q/14860665) – Michael Freidgeim Apr 12 '16 at 04:56

7 Answers7

22

I solved this problem with locking:

public static class IntegrationTestsSynchronization
{
    public static readonly object LockObject = new object();
}

[TestClass]
public class ATestCaseClass
{
    [TestInitialize]
    public void TestInitialize()
    {
        Monitor.Enter(IntegrationTestsSynchronization.LockObject);
    }

    [TestCleanup]
    public void TestCleanup()
    {
        Monitor.Exit(IntegrationTestsSynchronization.LockObject);
    }

    //test methods
}

// possibly other test cases

This can of course be extracted to a base test class and reused.

BartoszKP
  • 34,786
  • 15
  • 102
  • 130
12

I've fought for endless hours to make MSTest run in a single threaded mode on a large project that made heavy use of nHibernate and it's not-thread-safe (not a problem, it's just not) ISession.

We ended up more time writing code to support the multi-threaded nature of MSTest because - to the best of my and my teams knowledge - it is not possible to run MSTest in a single threaded mode.

BartoszKP
  • 34,786
  • 15
  • 102
  • 130
detroitpro
  • 3,853
  • 4
  • 35
  • 64
  • 2
    Thanks for the answer - I'll give up and switch back to NUnit – Paul Stovell Feb 18 '11 at 04:44
  • 2
    I'm confused. If your tests run sequentially, what does it matter whether they're on different threads or not? How does thread safety even matter in this case? – Merlyn Morgan-Graham Feb 18 '11 at 05:21
  • 7
    @Merlyn: It matters because of restrictions that may exist on components you don't own. As Paul mentioned, the WPF App class is a great example, running test cases against it may (will) fail because of thread ownership issues. This kind of defeats the purpose of the test because you are now running against an impossible scenario. (Your app will only ever have one App instance, so all threads will be the same). – A.R. Mar 29 '11 at 02:43
3

You can derive your test class from 

public class LinearTest
{
    private static readonly object SyncRoot = new object();

    [TestInitialize]
    public void Initialize()
    {
        Monitor.Enter(SyncRoot);
    }

    [TestCleanup]
    public void Cleanup()
    {
        Monitor.Exit(SyncRoot);
    }
}
mehmet mecek
  • 2,615
  • 2
  • 21
  • 25
2

We try hard to make out tests isolated from each other. Many of them achieve this by setting up the state of a database, then restoring it afterwards. Although mostly tests set up different data, with some 10,000 in a run there is a fair chance of a collision unless the code author of a test takes care to ensure its initial data is unique (ie doesn't use the same primary keys as another test doing something similar). This is, frankly, unmanageable, and we do get occasional test failures that pass second time around. I am fairly sure this is caused by collisions that would be avoided running tests strictly sequentially.

haughtonomous
  • 4,602
  • 11
  • 34
  • 52
2

The way to make an MSTest method run in single-threaded mode:

Nuget:

install-package MSTest.TestAdapter
install-package MSTest.TestFramework

In your test source on those methods that need to run while no other tests are running:

[TestMethod]
[DoNotParallelize]
public void myTest(){
    //
}
Wolfgang Grinfeld
  • 870
  • 10
  • 11
1

Whilst it is a cop out answer, I would actually encourage you to make your code thread-safe. The behaviour of MSTest is to ensure isolation as Richard has pointed out. By encountering problems with your unit tests you are proving that there could be some problems in the future.

You could ignore them, use NUnit, or deal with them and continue to use MSTest.

Mitch Denny
  • 2,095
  • 13
  • 19
  • 2
    In this case the problem is down to WPF - e.g., new MyApp(); new MyApp(); // second call throws, but each is thread bound – Paul Stovell Feb 19 '11 at 03:42
  • 2
    Ah I see the pain point now. Yeah, I guess if you are using an application framework that encourages the use of a single thread to function (UI frameworks are obviously in this category) then you might need to do some gymnastics to make it work. Just a though, what if you created a thread that you control and initialised all your WPF code on that thread, and marshalled all your test code across onto that thread? – Mitch Denny Feb 21 '11 at 00:05
  • Instead of doing new MyApp() you could try to abstract it out like so: http://social.msdn.microsoft.com/Forums/en-US/wpf/thread/786d5c06-0511-41c0-a6a2-5c4e44f8ffb6 – Anastasiosyal Nov 15 '11 at 15:58
0

I tried a bit of a different approach, because the underlying problem is that the names of the pipes are the problem. So I made a fakePipe, derived it from the one I use in the program. And named the pipe with the tests name.

[TestClass]
public class PipeCommunicationContractTests {
  private PipeDummy pipe;

  /// <summary>
  ///Gets or sets the test context which provides
  ///information about and functionality for the current test run.
  ///</summary>
  public TestContext TestContext { get; set; }

  [TestInitialize]
  public void TestInitialize() {
     pipe = new PipeDummy(TestContext.TestName);
     pipe.Start();
  }

  [TestCleanup]
  public void TestCleanup() {
  {
     pipe.Stop();
     pipe = null;
  }
   ...
  [TestMethod]
  public void CallXxOnPipeExpectResult(){
      var result = pipe.Xx();
      Assert.AreEqual("Result",result); 
  }
}

It appears to be a bit faster, since we can run on multiple cores and threads...

kfn
  • 620
  • 1
  • 7
  • 26