Dynamically create nunit test cases based off config file

Question

I'm looking for a way to dynamically create nunit test cases based off config file. In my App.config, I have a list of filenames and its expected operation. I would like to be able to add a new entry and have a test case be created dynamically based off that. My code is as follows:

App.config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <!--syntax is filename#operation where 0 = Valid File, 1 = Missing Column, 2 = Invalid File-->
    <add key="FILENAME_0" value="C:\Users\me\Desktop\file_0.csv#0" />
    <add key="FILENAME_1" value="C:\Users\me\Desktop\file_1.csv#1" />
    <add key="FILENAME_2" value="C:\Users\me\Desktop\file_2.csv#2" />
  </appSettings>
</configuration>

In my test fixture, I initialize a list of test case data, which is created by parsing the App.config file as follows:

[TestFixture]
public class MyTests
{
    public enum Operation
    {
        ValidFile = 0,
        MissingColumns = 1,
        InvalidFile = 2
    };

    /// <summary>
    /// Gets test case data with the file name and an enum of the expected operation.
    /// </summary>
    private static IEnumerable<TestCaseData> FilenameCases()
    {
        int i = 0;
        bool moreFilesToTest = true;
        var files = new Dictionary<string, Operation>();
        while (moreFilesToTest)
        {
            string filenameAndOp = ConfigurationManager.AppSettings[$"FILENAME_{i}"];
            if (filenameAndOp == null)
                moreFilesToTest = false;
            else
            {
                string filename = filenameAndOp.Split('#')[0];
                int operation = Int32.Parse(filenameAndOp.Split('#')[1]);
                Operation op = (Operation)operation;
                    files.Add(filename, op);
            }
            i++;
        }

        foreach (var pair in files)
            yield return new TestCaseData(pair.Key, pair.Value);
    }

    [Test, TestCaseSource("FilenameCases")]
    public void ShouldLoadFiles(string FILENAME, Operation expected)
    {
        // ... run test cases
    }    
}

This works, however, would need to rebuild the project each time in order for any changes in the App.config to be recognized. The code also only works if the TestCaseData is static. What is the correct way to have new test cases be created dynamically based off the configuration file.

It looks like NUnit doesn't support it, see the following discussion: https://github.com/nunit/nunit/issues/2514 However you can do it within your test: so the test itself ill read the files (either app.config or other config file). This of course will have the down (?) side of having only one test — Amittai Shapira, Jan 15 '18 at 20:45

score 2 · Accepted Answer · answered Jan 16 '18 at 21:28

After researching this post, I have found out that the issue is not with the code, but rather the the configuration file. As I was running the test via the command line, the test .dll was not referencing my App.config, but another file.

When I compiled my project, I noticed it produced a config file based off the name of the project. So a test project titled MyTestProject would produce a file called MyTestProject.dll.config. Therefore, when I made changes to that config file, my test cases reflected the changes, without recompiling the project.

Credit to both Charlie and Amittai Shapira, as NUnit doesn't support dynamic test cases and TestCaseSource needs to be static. However, the code I tried was the best work around, as the end result did produced dynamic test cases.

score 1 · Answer 2 · answered Jan 15 '18 at 20:44

TestCaseSource methods need to be static. There are a few ways around this if you need it but I see no problem with using static in your case. Just because a method is static, that doesn't mean it has to always return the same values.

There is no reason for you to recompile your code when you change the config file. However, you have to reload the test assembly in order for the new values in the config to be read. In some environments, like under VS, it may be easier to just recompile in order to force that to happen.

Essentially, I'd say you have already found the best way to do this, although my personal tendency would be to use a flat file rather than the config file.

Dynamically create nunit test cases based off config file

2 Answers2