SpecFlow, Selenium, NUnit, Parallelization: ChromeDriver Windows from two different NUnit Tests, keep having unexplained relation

Question

I have a selenium-webdriver-di.cs file like this:

using TechTalk.SpecFlow;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium;
using BoDi;
using System;
using System.IO;
using System.Text;
using System.Collections.Generic;

public class WebDriverHooks
{
    private readonly IObjectContainer container;
    private static Dictionary<string, ChromeDriver> drivers = new Dictionary<string, ChromeDriver>();
    private ScenarioContext _scenarioContext;
    private FeatureContext _featureContext;

    public WebDriverHooks(IObjectContainer container, ScenarioContext scenarioContext, FeatureContext featureContext)
    {
        this.container = container;
        _scenarioContext = scenarioContext;
        _featureContext = featureContext;
    }
    [BeforeFeature]
    public static void CreateWebDriver(FeatureContext featureContext)
    {
        Console.WriteLine("BeforeFeature");
        var chromeOptions = new ChromeOptions();
        chromeOptions.AddArguments("--window-size=1920,1080");
        drivers[featureContext.FeatureInfo.Title] = new ChromeDriver(chromeOptions);
    }
    [BeforeScenario]
    public void InjectWebDriver(FeatureContext featureContext)
    {
        if (!featureContext.ContainsKey("driver"))
        {
            featureContext.Add("driver", drivers[featureContext.FeatureInfo.Title]);
        }
    }
    [AfterFeature]
    public static void DeleteWebDriver(FeatureContext featureContext)
    {
        ((IWebDriver)featureContext["driver"]).Close();
        ((IWebDriver)featureContext["driver"]).Quit();
    }

And then in each of my .cs files that contains the step definitions, I have constructors like this:

using System;
using TechTalk.SpecFlow;
using NUnit.Framework;
using OpenQA.Selenium;
using System.Collections.Generic;
using PfizerWorld2019.CommunityCreationTestAutomation.SeleniumUtils;
using System.Threading;
using System.IO;

namespace PfizerWorld2019
{
    [Binding]
    public class SharePointListAssets
    {
        private readonly IWebDriver driver;
        public SharePointListAssets(FeatureContext featureContext)
        {
            this.driver = (IWebDriver)featureContext["driver"];
        }
    }
}

And then I'm using the driver variable in all the functions of the class. Lastly I have a file that I called Assembly.cs, where I put this for NUnit fixture level parallelization:

using NUnit.Framework;
[assembly: Parallelizable(ParallelScope.Fixtures)]

Which in SpecFlow's terms means parallelization on the Feature level(1 .feature file = 1 Nunit Test = 1 Nunit Fixture)

If I run my tests serially, they work fine.

But if I run 2 tests in parallel, any two tests, always something funny happens. For example: the first Chromedriver window tries to click an element and it clicks it if and only when the second Chromedriver window (that is running a different test) renders the exact same part of the website. But it sends the click to the correct window (the first one).

I have tried:

1. To use the IObjectContainer interface and then do containers[featureContext.FeatureInfo.Title].RegisterInstanceAs<IWebDriver>(drivers[featureContext.FeatureInfo.Title]) in the InjectWebDriver function
2. To use Thread.CurrentThread.ToString() instead of featureContext.FeatureInfo.Title for indexing
3. To do featureContext.Add(featureContext.FeatureInfo.Title + "driver", new ChromeDriver(chromeOptions) instead of drivers[featureContext.FeatureInfo.Title] = new ChromeDriver(chromeOptions); in the CreateWebDriver function.

I just do not understand what allows this "sharing". Since FeatureContext is used for everything related to driver spawning and destruction, how can two chromedrivers talk with each other?

Update: I tried the driver initialization & sharing like this:

[BeforeFeature]
public static void CreateWebDriver(FeatureContext featureContext)
{
    var chromeOptions = new ChromeOptions();
    chromeOptions.AddArguments("--window-size=1920,1080");
    chromeOptions.AddArguments("--user-data-dir=C:/ChromeProfiles/Profile" + uniqueIndex);
    WebdriverSafeSharing.setWebDriver(TestContext.CurrentContext.WorkerId, new ChromeDriver(chromeOptions));
}

and I made a webdriver-safe-sharing.cs file like this:

class WebdriverSafeSharing
{
    private static Dictionary<string, IWebDriver> webdrivers = new Dictionary<string, IWebDriver>();
    public static void setWebDriver(string driver_identification, IWebDriver driver)
    {
        webdrivers[driver_identification] = driver;
    }
    public static IWebDriver getWebDriver(string driver_identification)
    {
        return webdrivers[driver_identification];
    }
}

and then in each step definition function, I'm just calling WebdriverSafeSharing.getWebDriver(TestContext.CurrentContext.WorkerId) without any involvement of the FeatureContext. And I'm still getting the same issue. Notice how I'm doing chromeOptions.AddArguments("--user-data-dir=C:/ChromeProfiles/Profile" + uniqueIndex); because I'm also starting to not trust that chromedriver itself is thread safe. But even that did not help.

Update 2: It tried an even more paranoid webdriver-safe-sharing.cs class:

class WebdriverSafeSharing
{
    private static readonly Dictionary<string, ThreadLocal<IWebDriver>> webdrivers = new Dictionary<string, ThreadLocal<IWebDriver>>();
    private static int port = 7000;
    public static void setWebDriver(string driver_identification)
    {
        lock (webdrivers)
        {
            ChromeDriverService service = ChromeDriverService.CreateDefaultService();
            service.Port = port;
            var chromeOptions = new ChromeOptions();
            chromeOptions.AddArguments("--window-size=1920,1080");
            ThreadLocal<IWebDriver> driver =
            new ThreadLocal<IWebDriver>(() =>
            {
                return new ChromeDriver(service, chromeOptions);
            });
            webdrivers[driver_identification] = driver;
            port += 1;
            Thread.Sleep(1000);
        }
    }
    public static IWebDriver getWebDriver(string driver_identification)
    {
        return webdrivers[driver_identification].Value;
    }

It has a lock, a Threadlocal and unique port. It still does not work. Exact same issues.

Update 3: If I run two separate Visual Studio instances and I run 1 test for each, it works. Or by having two identical projects run side by side

and then selecting to run the tests in parallel:

Answer 1

The problem appears to be that you are stashing the IWebDriver object in the FeatureContext. The FeatureContext is created and reused for each scenario in a feature. While on the surface it appears safe for running tests in parallel using NUnit (which does not run scenarios in the same feature in parallel), my hunch is that this is not as safe as you think.

Instead, initialize and destroy the IWebDriver object with each scenario, rather than feature. The ScenarioContext should be thread safe, since it is created once for each scenario, and is only used for one scenario. I would recommend using dependency injection instead:

[Binding]
public class WebDriverHooks
{
    private readonly IObjectContainer container;

    public WebDriverHooks(IObjectContainer container)
    {
        this.container = container;
    }

    [BeforeScenario]
    public void CreateWebDriver()
    {
        var driver = // Initialize your web driver here

        container.RegisterInstanceAs<IWebDriver>(driver);
    }

    [AfterScenario]
    public void DestroyWebDriver()
    {
        var driver = container.Resolve<IWebDriver>();

        // Take screenshot if you want...

        // Dispose of the web driver
        driver.Dispose();
    }
}

Then add a constructor argument to your step definition classes to pass the IWebDriver:

[Binding]
public class FooSteps
{
    private readonly IWebDriver driver;

    public FooSteps(IWebDriver driver)
    {
        this.driver = driver;
    }

    // step definitions...
}

Answer 2

The reason was that all the selenium API actions were wrapped in static methods written by me. It was a class in a file that I wrote many weeks ago for code re-usability. However not being used into working with parallel programming in C#, I was honestly not aware anymore that these methods were declared static. I am now running 20 parallel workers on Selenium Grid.

However I'm placing here some important notes to be aware of, if one faces parallelization issues with NUnit, SpecFlow and Selenium

• The initialization of the WebDriver must be done in the [BeforeFeauture]-bound method if the goal is feature-level and not scenario-level parallelization.
• The initialization of the WebDriver must be thread safe. What I did is that I used a static Dictionary that is indexed by the FeatureContext.FeatureInfo.Title that contains the WebDrivers
• chromedriver is thread safe. There is no need unique data-dir folders or unique ports or unique chromedriver file names. The --headless and --no-sandbox arguments that one might be interested in have not caused me any issues with parallelization (either with Selenium Grid or simple single multi-core machine parallelization). Basically don't blame the chromedriver.
• For injecting the webdriver, do use the IObjectContainer interface in the [BeforeScenario]-bound method. It is great and safe.
• Dispose the driver in the [AfterFeature]-bound method with driver.Dispose() so that you don't have zombie processes. This helped me with Selenium Grid, because when I was using driver.Close() and driver.Quit(), the nodes would not kill the processes after the latter were done.
• For NUnit with [assembly: Parallelizable(ParallelScope.Fixtures)] enabled, all the scenarios within a .feature file run within the same FeatureContext and therefore the scenarios are FeatureContext-safe. This means that you can trust the FeatureContext for sharing data between scenarios within the same .feature file.
• The [BeforeFeature] hook is being called only once per .feature file (as one would logically assume). So if you have x .feature files, the hook will be called x times during a test run.
• As the Specflow Documentation says, for feature-level parallelization either NUnit or xUnit should be used as the test framework. However the NUnit has the benefit that it provides out of the box ordering for the tests, by alphabetical sort order. This is particularly useful if you want to have two scenarios run in sequence within the same feature file. i.e. putting a number in front of each scenario title will ensure their order during execution. xUnit does not support this natively and it looked like a difficult goal to achieve from searching around.
• Specflow is more "friendly" in terms of parallelization, for scenario-level parallelization. This is why the Specflow+ Runner test framework by SpecFlow runs scenario levels in parallel. it looks like the whole philosophy of SpecFlow (I wouldn't say BDD yet) is to have independent scenarios. This doesn't mean of course that you cannot have very nice feature-level parallelization by using the other test frameworks. Just putting it out there as a heads up for someone reading this while drafting a strategy for writing feature files.