0

I have posted a similar question before but I am still looking for a suitable resolution for this issue in relation to automated testing with Selenium.

To re-cap, because of performance issues on our test servers and attached services we sometimes have tests that fail first time but will pass on a rerun.

Previously in RubyMine the retry could be programmed using something like this example:

default: RDEE_BROWSER=chrome --no-source --color --format pretty --format html --out testresults/<%= Time.now.strftime("%Y%m%d-%H%M%S") %>-pu_automation.html --retry 1 --tags @current_tests

Where the retry 1 option would force a retry if the test failed, this usually passed on second try.

Now we have switched to Visual Studio/C#/Selenium with Nunit and Specflow and here is a typical example In the feature file

Scenario Outline: Validate that the search locates the required item
    Given I browse to the homepage
    When I search for "search"
    Then  I am presented with a list of items for "search"

Which links to the step definition file

[Given(@"I browse to the homepage")]
public void GivenIBrowseToTheHomepage()
{
    Visit();
}

[When(@"I search for ""(.*)""")]
public void WhenISearchFor(string SearchText)
{
    StartaSearch(SearchText);
}


[Then(@"I am presented with a list of items for ""(.*)""")]
public void ThenIAmPresentedWithAListOfItemsFor(string items)
{
    AssertIsTrue(ValidateItemList(items));
}

Which in turn links to the page class where the actual work is started

We also have the hooks class as follows

public sealed class Hooks : SharedClass
{
    private readonly ScenarioContext _scenarioContext;

    public Hooks(ScenarioContext scenarioContext)
    {
        _scenarioContext = scenarioContext;
    }

    [BeforeScenario]
    public void BeforeScenario()
    {
        _test = _extent.CreateTest(_scenarioContext.ScenarioInfo.Title.ToString());
    }

    [AfterScenario]
    public void AfterScenario()
    {
        var status = _scenarioContext.ScenarioExecutionStatus.ToString();
        if (status != "OK")
        {
            var stackTrace = "<pre>" + _scenarioContext.TestError.StackTrace + "</pre>";
            var errorMessage = GetErrorMessage();
            WriteToReport(stackTrace + errorMessage, Status.Fail);
        }
        Driver.Manage().Cookies.DeleteAllCookies();
    }

    //preview
    [BeforeFeature]
    [Scope(Tag = "demo_tests")]
    public static void BeforeFeaturePreview(FeatureContext featureContext)
    {
        BeforeFeature(featureContext.FeatureInfo.Title);
    }

    public static void BeforeFeature(string title)
    {
        ResetReportVariables(TestUrls.DomainSpice, title);
        var options = new ChromeOptions();
        options.AddArguments("disable-browser-side-navigation");
        options.AddArguments("disable-infobars");
        options.AddArgument("ignore-certificate-errors");
        options.AddArgument("ignore-ssl-errors");
        options.AddArgument("disable-popup-blocking");
        options.AddArguments("start-maximized");
        options.AddArguments("no-sandbox");
        options.SetLoggingPreference(LogType.Browser, LogLevel.All);
        Driver = new ChromeDriver(options);
    }

    [AfterFeature]
    public static void AfterFeature()
    {
        try
        {
            Driver.Manage().Cookies.DeleteAllCookies();
        }
        catch (Exception ex)
        {
            Console.WriteLine("Already logged out - ignore : {0}", ex);
        }
        _extent.Flush();
        try
        {
            Driver.Close();
            Driver.Quit();
        }
        catch (Exception ex)
        {
            Console.WriteLine("Error: " + ex);
        }
    }

We are using NUnit.Framework.Assert to mark a test as pass or fail but now can we enable a retry option in this example project? Is Nunit the wrong option? Would MsTest be a better option? Or is the something else better suited?

These are the packages we currently use

<package id="Cucumber.Messages" version="6.0.1" targetFramework="net472" />
<package id="DotNetSeleniumExtras.WaitHelpers" version="3.11.0" targetFramework="net472" />
<package id="ExtentReports" version="4.1.0" targetFramework="net472" />
<package id="Gherkin" version="6.0.0" targetFramework="net472" />
<package id="NUnit" version="3.12.0" targetFramework="net472" />
<package id="NUnit3TestAdapter" version="3.17.0" targetFramework="net472" />
<package id="Selenium.Support" version="3.141.0" targetFramework="net472" />
<package id="Selenium.WebDriver" version="3.141.0" targetFramework="net472" />
<package id="SpecFlow" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.CustomPlugin" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.NUnit" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.Tools.MsBuild.Generation" version="3.4.8" targetFramework="net472" />

Thanks in advance

Greg Burghardt
  • 17,900
  • 9
  • 49
  • 92
Kev
  • 121
  • 3
  • 20
  • 1
    If you were using NUnit alone, I'd simply tell you to use the RetryAttribute. Instead, you need some way to tell SpecFlow to use it. Unfortunately, I don't know SpecFlow so I can't answer. Suggest adding SpecFlow tag to this, since that's actually the main issue. – Charlie Oct 14 '20 at 15:40
  • Does this answer your question? [Is it possible to run same specflow test again based on outcome?](https://stackoverflow.com/questions/40735099/is-it-possible-to-run-same-specflow-test-again-based-on-outcome) – Greg Burghardt Oct 14 '20 at 19:45
  • See also [SpecFlow - Retry failed tests](https://stackoverflow.com/q/21278338/3092298). – Greg Burghardt Oct 14 '20 at 19:45

0 Answers0