I have posted a similar question before but I am still looking for a suitable resolution for this issue in relation to automated testing with Selenium.
To re-cap, because of performance issues on our test servers and attached services we sometimes have tests that fail first time but will pass on a rerun.
Previously in RubyMine the retry could be programmed using something like this example:
default: RDEE_BROWSER=chrome --no-source --color --format pretty --format html --out testresults/<%= Time.now.strftime("%Y%m%d-%H%M%S") %>-pu_automation.html --retry 1 --tags @current_tests
Where the retry 1 option would force a retry if the test failed, this usually passed on second try.
Now we have switched to Visual Studio/C#/Selenium with Nunit and Specflow and here is a typical example In the feature file
Scenario Outline: Validate that the search locates the required item
Given I browse to the homepage
When I search for "search"
Then I am presented with a list of items for "search"
Which links to the step definition file
[Given(@"I browse to the homepage")]
public void GivenIBrowseToTheHomepage()
{
Visit();
}
[When(@"I search for ""(.*)""")]
public void WhenISearchFor(string SearchText)
{
StartaSearch(SearchText);
}
[Then(@"I am presented with a list of items for ""(.*)""")]
public void ThenIAmPresentedWithAListOfItemsFor(string items)
{
AssertIsTrue(ValidateItemList(items));
}
Which in turn links to the page class where the actual work is started
We also have the hooks class as follows
public sealed class Hooks : SharedClass
{
private readonly ScenarioContext _scenarioContext;
public Hooks(ScenarioContext scenarioContext)
{
_scenarioContext = scenarioContext;
}
[BeforeScenario]
public void BeforeScenario()
{
_test = _extent.CreateTest(_scenarioContext.ScenarioInfo.Title.ToString());
}
[AfterScenario]
public void AfterScenario()
{
var status = _scenarioContext.ScenarioExecutionStatus.ToString();
if (status != "OK")
{
var stackTrace = "<pre>" + _scenarioContext.TestError.StackTrace + "</pre>";
var errorMessage = GetErrorMessage();
WriteToReport(stackTrace + errorMessage, Status.Fail);
}
Driver.Manage().Cookies.DeleteAllCookies();
}
//preview
[BeforeFeature]
[Scope(Tag = "demo_tests")]
public static void BeforeFeaturePreview(FeatureContext featureContext)
{
BeforeFeature(featureContext.FeatureInfo.Title);
}
public static void BeforeFeature(string title)
{
ResetReportVariables(TestUrls.DomainSpice, title);
var options = new ChromeOptions();
options.AddArguments("disable-browser-side-navigation");
options.AddArguments("disable-infobars");
options.AddArgument("ignore-certificate-errors");
options.AddArgument("ignore-ssl-errors");
options.AddArgument("disable-popup-blocking");
options.AddArguments("start-maximized");
options.AddArguments("no-sandbox");
options.SetLoggingPreference(LogType.Browser, LogLevel.All);
Driver = new ChromeDriver(options);
}
[AfterFeature]
public static void AfterFeature()
{
try
{
Driver.Manage().Cookies.DeleteAllCookies();
}
catch (Exception ex)
{
Console.WriteLine("Already logged out - ignore : {0}", ex);
}
_extent.Flush();
try
{
Driver.Close();
Driver.Quit();
}
catch (Exception ex)
{
Console.WriteLine("Error: " + ex);
}
}
We are using NUnit.Framework.Assert to mark a test as pass or fail but now can we enable a retry option in this example project? Is Nunit the wrong option? Would MsTest be a better option? Or is the something else better suited?
These are the packages we currently use
<package id="Cucumber.Messages" version="6.0.1" targetFramework="net472" />
<package id="DotNetSeleniumExtras.WaitHelpers" version="3.11.0" targetFramework="net472" />
<package id="ExtentReports" version="4.1.0" targetFramework="net472" />
<package id="Gherkin" version="6.0.0" targetFramework="net472" />
<package id="NUnit" version="3.12.0" targetFramework="net472" />
<package id="NUnit3TestAdapter" version="3.17.0" targetFramework="net472" />
<package id="Selenium.Support" version="3.141.0" targetFramework="net472" />
<package id="Selenium.WebDriver" version="3.141.0" targetFramework="net472" />
<package id="SpecFlow" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.CustomPlugin" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.NUnit" version="3.4.8" targetFramework="net472" />
<package id="SpecFlow.Tools.MsBuild.Generation" version="3.4.8" targetFramework="net472" />
Thanks in advance