Get test reporter to break down test when passing func parameters to Theory

Question

I am passing Func parameters to a unit test theory. See Pass complex parameters to [Theory] for the gist of what I'm doing.

The problem I'm having is that the test reporting doesn't break down the results individually. Here is a small example to illustrate. The production theory would have a lot more cases.

public class TestReporting {
  public static IEnumerable<object[]> BoolTestData() {
    yield return new object[] { true, true };
    yield return new object[] { true, false };
  }
  [Theory, MemberData(nameof(BoolTestData))]
  public void CheckBoolEquality(bool b1, bool b2) {
    Assert.Equal(b1, b2);
  }

  public static IEnumerable<(string, Func<string, string>, string)> RawTestData() {
    yield return ("Hello", str => str.Substring(3), "lo");
    yield return ("World", str => str.Substring(0, 4), "worl");
  }
  public static IEnumerable<object[]> StringTestData() {
    return RawTestData().Select(vt => new object[] { vt.Item1, vt.Item2, vt.Item3 });
  }

  [Theory, MemberData(nameof(StringTestData))]
  public void RunStringTest(string input, Func<string, string> func, string expectedOutput) {
    var output = func(input);
    Assert.Equal(expectedOutput, output);
  }
}

Test run output -- paraphrased as it doesn't seem to be possible to copy-paste just the top-level info

Failed Tests(2)
  CheckBoolEquality(b1: True, b2: False)
  RunStringTest

Passed Tests(1)
  CheckBoolEquality(b1: True, b2: True)

Notice that in the bool case, it broke out my test into two tests, a passing claim that true=true, and a failing claim that true=false. That's what I want.

But in the string case, it presumably doesn't know how to describe the Func, so it just gives up and says the whole thing failed.

I'd be perfectly happy to have it just report the parts it can, i.e. something like this:

Failed Tests(2)
  CheckBoolEquality(b1: True, b2: False)
  RunStringTest("World")

Passed Tests(2)
  CheckBoolEquality(b1: True, b2: True)
  RunStringTest("Hello")

Is that possible?

score 0 · Answer 1 · answered May 16 '17 at 15:55

This improves the reporting -- you see the results inside the failing string test, at least:

public class TestReporting {
  public static IEnumerable<object[]> BoolTestData() {
    yield return new object[] { true, true };
    yield return new object[] { true, false };
  }
  [Theory, MemberData(nameof(BoolTestData))]
  public void CheckBoolEquality(bool b1, bool b2) {
    Assert.Equal(b1, b2);
  }

  public static IEnumerable<(string, Expression<Func<string, string>>, string)> RawTestData() {
    yield return ("Hello", str => str.Substring(3), "lo");
    yield return ("World", str => str.Substring(0, 4), "worl");
  }
  public static IEnumerable<object[]> StringTestData() {
    return RawTestData().Select(vt => new object[] { vt.Item1, vt.Item2, vt.Item3 });
  }

  [Theory, MemberData(nameof(StringTestData))]
  public void RunStringTest(string input, Expression<Func<string, string>> func, string expectedOutput) {
    var output = func.Compile()(input);
    Assert.Equal(expectedOutput, output);
  }

}

Get test reporter to break down test when passing func parameters to Theory

1 Answers1