How to parallelize a Data-Driven unit test in Visual Studio 2010?

Question

I know regular MS-Test unit tests can be parallelized on a multi-core machine (with caveats of course) by specifying parallelTestCount attribute in the .testresults file in the test solution. Like this,

<Execution parallelTestCount="1">
    <TestTypeSpecific />
    <AgentRule name="Execution Agents"></AgentRule>
</Execution>

More at MSDN: Executing Unit Tests in parallel on a multi-CPU/core machine

However, I have a data-driven test, something like this, this is just one test, but the input comes in from a csv and runs 1000s of records through the same test.

[DeploymentItem("InputDataRows.csv"), Timeout(37800000), DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\InputDataRow.csv", "InputDataRow#csv", DataAccessMethod.Sequential)]                
[TestMethod]
public void RunProcessing()
{
    int userId = Convert.ToInt32(TestContext.DataRow[0].ToString());
    int connId = Convert.ToInt32(TestContext.DataRow[1].ToString());
    string xml = TestHelper.GetDataFromDb(userId, connId);
    a = doStuffA(xml); 
    b = doStuffB(xml);
    Assert.IsTrue(a == b);
}

Because this is a slow process, I am looking at parallelizing this unit test.

The Sequential enum on the attribute is just the way it accesses data, the other option is Random, which is still serial and not parallel.

What version of .Net are you using? Also, can you provide some details on how the TestContext is being set? — Paul Tsai, Jan 03 '17 at 20:21

score 1 · Answer 1 · edited Jul 28 '17 at 15:36

In order to parallelize this unit test, you'll need doStuffA() and doStuffB() to be able to operate on a subset of the data (e.g. a chunk or even a single row of your csv at a time). If you can refactor your methods to behave in this way, you can utilize tasks or a parallel foreach loop so that this test executes in parallel. Suppose your methods were refactored to handle a row of your csv, you could do something like this:

int userId = Convert.ToInt32(TestContext.DataRow[0].ToString());
int connId = Convert.ToInt32(TestContext.DataRow[1].ToString());
string xml = TestHelper.GetDataFromDb(userId, connId);
var rows = xml.Split('\n');

Parallel.ForEach(rows, (row) =>
{
    var a = doStuffOnRowA(row);
    var b = doStuffOnRowB(row);
    Assert.AreEqual(a, b);
});

score 1 · Answer 2 · answered Dec 07 '17 at 22:39

This might seem a bit complex, but hear me out. There is a limitation in MSTest that you cannot actually run data-driven tests in parallel. What I have done in the past to get around this is to create a "custom tool" in visual studio.

https://msdn.microsoft.com/en-us/library/bb166508.aspx

OR

https://msdn.microsoft.com/en-us/library/bb166817.aspx

The custom tool that we created did the following:

Split out the csv into multiple csv files with only one row each.
Generate an individual test for each of the newly generated csvs.

When these tests were generated, we put specific test attributes on them, so we could specify to only run the tests with that attribute.

This sounds kind of over the top, but if you do a good job building the custom tool, it's actually a very smooth process.

Garett Roberts · Answer 3 · 2018-10-09T20:55:10.927

1

If there is no data being changed in the xml (Or the methods are not modifying the same parts of the xml) then you could do..

var numCompleted = 0;
var a = Task.Run(() => { doStuffOnRowA(xml); });
var b = Task.Run(() => { doStuffOnRowB(xml); });
Task.WaitAll(new Task[2] { a, b });

This might not run if you copy and paste, kind of pseudo code. Not exactly Parallel, but will at least run around the same time!

edited Oct 09 '18 at 20:55

answered Sep 10 '18 at 21:32

Garett Roberts

31
1
5

1

You should better use this function of the framework when you are waiting for multiple tasks Task.WaitAll https://msdn.microsoft.com/en-us/library/windows/apps/dd270695(v=vs.105).aspx – David Oct 09 '18 at 13:36
Thank you, I just changed it! – Garett Roberts Oct 09 '18 at 20:55

score 1 · Answer 4 · answered Jul 04 '19 at 05:30

1

Parallel execution of data driven tests is not supported. Please see here: RFC 004 - In-assembly Parallel Execution

answered Jul 04 '19 at 05:30

pvlakshm

1,365
7
7

For someone looking for this, see very last line in document: "4. Execution of data driven tests will not be parallelized - i.e. parallelizing over DataRow attributes is not supported." – DannyMeister Jan 12 '22 at 20:50

score -2 · Answer 5 · answered Aug 14 '13 at 00:57

-2

As as I know: individual data rows in a test are NOT run in parallel. But if you have multiple unit tests, they do run in parallel.

answered Aug 14 '13 at 00:57

Venkata Sadineni

1

How to parallelize a Data-Driven unit test in Visual Studio 2010?

5 Answers5