Rerunning the failed scenario using Maven/Cucumber/Serenity

Question

Has anyone used maven surefire plug-in or any other mechanism for rerunning failed scenario. I am using Cucumber with Serenity and Maven. I tried below different ways in order to rerun failed scenario without any manual intervention
For example: If out of 5 test cases 2 test cases are getting error out then automatically my script should execute these 2 failed scenario before generating final serenity report
1. Maven surefire
I have added below lines in pom.xml file

<properties>
    <failsafe.rerunFailingTestsCount>2</failsafe.rerunFailingTestsCount>
    <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
</properties>

OR

        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <includes>
                    <include>**/*.java</include>
                </includes>
                <argLine>-Xmx512m</argLine>
                <systemPropertyVariables>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver>                        
                </systemPropertyVariables>
                <systemProperties>
                    <webdriver.driver>${webdriver.driver}</webdriver.driver> 
                    <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                    <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                </systemProperties>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

Then I tried to execute below command (my scenario is annoted by @RunThis tag)

mvn -Dcucumber.options="--tags @RunThis" -Dfailsafe.rerunFailingTestsCount=2 test

After executing above command I am expecting that failed scenario should get executed again but it is not re-executing

2. Generating rerun.txt using cucumber formatter and executing it
As per another solution, I have generate rerun.txt using cucumber formatter. Scenario which are getting error out will be listed down in rerun.txt which should be then picked up by second runner class and execute those failed scenario.
I am able to generated rerun.txt successfully but second runner class is neither getting executed nor I am getting any kind of error

Runnerclass 1

 @RunWith(CucumberWithSerenity.class)
    @CucumberOptions(
            features = "src/test/resources/features",
            tags = "@RunThis",
            monochrome = true
            ,plugin = { "json:target/cucumber-report-composite.json", "pretty",
                "html:target/cucumber/","rerun:target/rerun.txt" })
    public class AcceptanceTest {   

    }

RunnerClass 2

@RunWith(CucumberWithSerenity.class)
    @CucumberOptions(strict = true, 
      glue = { "src/test/resources/features" }, 
      features = { "@target/rerun.txt" }, 
      plugin = { "json:target/cucumber-report-composite.json", "pretty",
        "html:target/cucumber/"})
public class AcceptanceTest2 {

}

3. Using @ExtendCucumberOptions
As per one more solution we can add below dependancy on pom.xml file

<dependency>
    <groupId>com.github.mkolisnyk</groupId>
    <artifactId>cucumber-runner</artifactId>
    <version>1.3.1</version>
</dependency>

Then our runner class should be like this

@RunWith(CucumberWithSerenity.class)
@ExtendedCucumberOptions(jsonReport = "target/cucumber.json",
retryCount = 3,
jsonUsageReport = "target/cucumber-usage.json",
outputFolder = "target")
@CucumberOptions(
        features = "src/test/resources/features",
        tags = "@RunThis",
        monochrome = true
        ,plugin = { "json:target/cucumber-report-composite.json", "pretty",
                "html:target/cucumber/","rerun:target/rerun.txt" })
public class AcceptanceTest {


}

Then I am executing below command
mvn -Dcucumber.options="--tags @RunThis"
again I could not see that failed scenario is getting re-executed

complete POM.xml file

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>GROUPNAME</groupId>
    <artifactId>ARTIFACTNAME</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>Serenity project using Cucumber and WebDriver</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <failsafe.rerunFailingTestsCount>2</failsafe.rerunFailingTestsCount>
        <surefire.rerunFailingTestsCount>2</surefire.rerunFailingTestsCount>
        <serenity.version>1.6.3</serenity.version>
        <serenity.cucumber.version>1.5.9</serenity.cucumber.version>
        <serenity.maven.version>1.2.1</serenity.maven.version>
        <webdriver.driver>chrome</webdriver.driver>
    </properties>

    <repositories>
        <repository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray</name>
            <url>http://jcenter.bintray.com</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray-plugins</name>
            <url>http://jcenter.bintray.com</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencies>
        <!-- <dependency> <groupId>net.serenity-bdd.maven.plugins</groupId> <artifactId>serenity-maven-plugin</artifactId> 
            <version>1.5.0-rc.1</version> </dependency> -->

        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-core</artifactId>
            <version>${serenity.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-cucumber</artifactId>
            <version>${serenity.cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.lambdaj</groupId>
            <artifactId>lambdaj</artifactId>
            <version>2.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20171018</version>
        </dependency>
        <dependency>
            <groupId>com.qpid.automation</groupId>
            <artifactId>AutomationToolKit</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>com.github.mkolisnyk</groupId>
            <artifactId>cucumber-runner</artifactId>
            <version>1.3.1</version>
        </dependency>

    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <!-- <skip>true</skip> -->
                    <argLine>-Dfile.encoding="UTF-8"</argLine>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <includes>
                        <include>**/*.java</include>
                    </includes>
                    <argLine>-Xmx512m</argLine>
                    <systemPropertyVariables>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver>                        
                    </systemPropertyVariables>
                    <systemProperties>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver> 
                        <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                        <surefire.rerunFailingTestsCount>${surefire.rerunFailingTestsCount}</surefire.rerunFailingTestsCount>
                    </systemProperties>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <!-- Generate the test reports after the integration tests -->
            <plugin>
                <groupId>net.serenity-bdd.maven.plugins</groupId>
                <artifactId>serenity-maven-plugin</artifactId>
                <version>${serenity.version}</version>
                <executions>
                    <execution>
                        <id>serenity-reports</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>aggregate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

score 2 · Accepted Answer · answered Mar 08 '18 at 08:01

I could achieve the rerun using below code.

@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
        features = "src/test/resources/features", tags = "@Smoke1",
        monochrome = true,
        plugin = {"pretty", "html:target/cucumber-reports", "json:target/cucumber.json","rerun:rerun.txt"})
public class AcceptanceTest {

}

I just execute below command, after adding rerun in above plug in(@CucumberOptions)

mvn clean verify -Dcucumber.options="--tags @Smoke1"

and it automatically re-execute failed scenario. I no need to write another runner class and run it manually.

My pom.xml is:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>GROUPID</groupId>
    <artifactId>ARTIFACTID</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>Serenity project using Cucumber and WebDriver</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <serenity.version>1.9.2</serenity.version>
        <serenity.cucumber.version>1.9.3</serenity.cucumber.version>
        <serenity.maven.version>1.8.21</serenity.maven.version>
        <webdriver.driver>chrome</webdriver.driver>
    </properties>

    <repositories>
        <repository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray</name>
            <url>http://jcenter.bintray.com</url>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
            <id>central</id>
            <name>bintray-plugins</name>
            <url>http://jcenter.bintray.com</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencies>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-core</artifactId>
            <version>${serenity.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-cucumber</artifactId>
            <version>${serenity.cucumber.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.1.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>1.7.0</version>
        </dependency>
        <dependency>
            <groupId>com.googlecode.lambdaj</groupId>
            <artifactId>lambdaj</artifactId>
            <version>2.3.3</version>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20171018</version>
        </dependency>
        <dependency>
            <groupId>com.qpid.automation</groupId>
            <artifactId>AutomationToolKit</artifactId>
            <version>1.0.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>
                <configuration>
                    <!-- <skip>true</skip> -->
                    <argLine>-Dfile.encoding="UTF-8"</argLine>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.18.1</version>
                <configuration>
                    <includes>
                        <include>**/*.java</include>
                    </includes>
                    <argLine>-Xmx512m</argLine>
                    <systemPropertyVariables>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver>
                    </systemPropertyVariables>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <!-- Generate the test reports after the integration tests -->
            <plugin>
                <groupId>net.serenity-bdd.maven.plugins</groupId>
                <artifactId>serenity-maven-plugin</artifactId>
                <version>${serenity.version}</version>
                <executions>
                    <execution>
                        <id>serenity-reports</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>aggregate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Two questions: `1.` mvn clean verify -Dcucumber.options="--tags @Smoke1" in this command where did you mention to run failed scenarios `2.` my target folder doesn't have cucumber.json file, where did you get that? Please help, I also need this fix. — paul, Apr 09 '19 at 12:28
@paul, Line '"rerun:rerun.txt"' in plug in will take care of rerunning failed scenario.
**How it Works** - When you first time executes your scenarios, let's say 2 scenarios out of total 10 scenarios get error out. These two scenario will get noted down in rerun.txt. After execution completes successfully, cucumber will check for rerun.txt and if it contains any scenario then it will run those scenario again — SachinB, Apr 24 '19 at 07:05
@Python_Novice, No need to make any change in Jenkins job, you will execute your suite using Jenkins as usual. When any of scenario get error out then corresponding entry will get added in 'rerun.txt', after suite get executed completely and if there any entry got added in 'rerun.txt' then this particular scenario will get re-executed — SachinB, Sep 18 '19 at 08:49
@SachinB - Don't know what I was thinking at that day, but today I understood the analogy very clearly. And it worked like a charm. Thanks you made my day! — Python_Novice, Oct 16 '19 at 14:51
@SachineB - One thing, can we have some kind of logging for this? Like suppose after first run, system says like ''Rerunning failures for one more time ..." ? If you can help me this please. — Python_Novice, Oct 16 '19 at 15:28
@SachinB its running now, but it's not rerunning failed scenarios. I could see rerun.txt is generated with details of failed tests but it's not picking up that automatically — Python_Novice, Oct 16 '19 at 16:08
@paul did you get it working for you? I can write the failed tests in rerun.txt but it's get re executed automatically as mentioned in the solution. — Python_Novice, Oct 17 '19 at 12:52
@SachinB In my case I also see `rerun.txt` failing tests, in the command-line logs there is no evidence to say that it had rerun. — Shabar, May 19 '20 at 05:32

Rerunning the failed scenario using Maven/Cucumber/Serenity

1 Answers1

Linked