1

Allure report on testng + cucumber framework gives a duplicate result report for cucumber tests.

Problem: Whenever I run a local allure report >allure serve allure-report, I got a duplicate report on cucumber tests, one using the proper notations and visualizations for allure-cucumber tags and the other with a surefire path and no correct consumptions of tags/information.

Correct report

Bad report

In both images there is the left index for suites and there, both appears, each image is for each test expanded.

Scenario: My project implements Java + maven + testng + cucumber and using allure as reporting system, to execute maven-surefire-plugin and to run have the following maven command: clean test -Ddriver=chrome -Dmaximize=false -DthreadCount=3 -Dexecutor=jenkins -Dtimeout=10 -PCucumber using a testng.xml file to run the cucumber test cases. Also this project has testng test cases, I'm trying to implement both systems for automation in the same project, TestNG cases are correctly logged once, but only cucumber ones got this kind of double reporting.

This bad report doesn't consumes @Step allure notation for reporting so the only possibility that remains for me is that somehow testng/surefire is indexing/consuming the Cucumber test cases definition (parameters, setuUp, tearDown and result) but none of the allure annotations, because of that would discard any class building as the problem and I'm trying to figure out the problem on the pom but open to any solution.

More info:

  • PageObject models only contains @Step annotations
  • DriverManager class contains a ThreadLocal to setup, fetch and tearDown drivers for each test but don't have any ncucumber or testng notation for hierarchy of execution, this methods are called by BaseTest and CucumberSupport respective clases with respective notation for each dependency.

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.seleniumTest</groupId>
    <artifactId>SeleniumTest</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>SeleniumTest</name>
    <description>POC for selenium testing project implementation.</description>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <driver>chrome</driver>
        <maximize>false</maximize>
        <executor>jenkins</executor>
        <timeout>10</timeout>
        <threadCount>3</threadCount>

        <webdrivermanager-version>5.3.3</webdrivermanager-version>
        <testng-version>7.7.1</testng-version>
        <log4j-core-version>2.20.0</log4j-core-version>
        <selenium-java-version>4.9.1</selenium-java-version>
        <allure-testng-version>2.22.0</allure-testng-version>
        <jul-to-slf4j-version>2.0.7</jul-to-slf4j-version>
        <cucumber-version>7.12.0</cucumber-version>
        <gherkin-version>26.0.1</gherkin-version>
        <allure-cucumber7-jvm-version>2.21.0</allure-cucumber7-jvm-version>
        <maven-compiler-plugin-version>3.11.0</maven-compiler-plugin-version>
        <maven-surefire-plugin-version>3.0.0-M7</maven-surefire-plugin-version>
        <aspectjweaver-version>1.9.8.RC2</aspectjweaver-version>
        <allure-maven-version>2.22.0</allure-maven-version>


    </properties>

    <dependencies>
        <!-- WebDriver automated manager -->
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>${webdrivermanager-version}</version>
        </dependency>

        <!-- Testing framework -->
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>${testng-version}</version>
        </dependency>

        <!-- Logger dependency -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>${log4j-core-version}</version>
        </dependency>

        <!-- Selenium framework for web testing -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium-java-version}</version>
        </dependency>

        <!-- Reporting framework -->
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-testng</artifactId>
            <version>${allure-testng-version}</version>
        </dependency>

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jul-to-slf4j</artifactId>
            <version>${jul-to-slf4j-version}</version>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-testng</artifactId>
            <version>${cucumber-version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>gherkin</artifactId>
            <version>${gherkin-version}</version>
            <scope>test</scope>
        </dependency>
        
        <dependency>
            <groupId>io.qameta.allure</groupId>
            <artifactId>allure-cucumber7-jvm</artifactId>
            <version>${allure-cucumber7-jvm-version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <profiles>
        <profile>
            <id>TestNG</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>${maven-surefire-plugin-version}</version>
                        <configuration>
                            <suiteXmlFiles>
                                <suiteXmlFile>config/testng.xml</suiteXmlFile>
                            </suiteXmlFiles>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
        <profile>
            <id>Cucumber</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>${maven-surefire-plugin-version}</version>
                        <configuration>
                            <includes>
                                <include>**/*CucumberRunnerTests.java</include>
                            </includes>
                            <properties>
                                <property>
                                    <name>dataproviderthreadcount</name>
                                    <value>${threadCount}</value>
                                </property>
                            </properties>
                            <argLine>
                                -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectjweaver-version}/aspectjweaver-${aspectjweaver-version}.jar"
                            </argLine>
                        </configuration>
                        <dependencies>
                            <dependency>
                                <groupId>org.aspectj</groupId>
                                <artifactId>aspectjweaver</artifactId>
                                <version>${aspectjweaver-version}</version>
                            </dependency>
                        </dependencies>
                    </plugin>
                </plugins>
            </build>
        </profile>
        <profile>
            <id>TestNG+Cucumber</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>${maven-surefire-plugin-version}</version>
                        <configuration>
                            <properties>
                                <property>
                                    <name>dataproviderthreadcount</name>
                                    <value>${threadCount}</value>
                                </property>
                            </properties>
                            <suiteXmlFiles>
                                <suiteXmlFile>config/cucumber.xml</suiteXmlFile>
                                <suiteXmlFile>config/testng.xml</suiteXmlFile>
                            </suiteXmlFiles>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <reporting>
        <plugins>
            <plugin>
                <groupId>io.qameta.allure</groupId>
                <artifactId>allure-maven</artifactId>
                <version>${allure-maven-version}</version>
                <configuration>
                    <propertiesFilePath>test/resources/allure.properties</propertiesFilePath>
                </configuration>
            </plugin>
        </plugins>
    </reporting>

</project>

Repo for reference: https://github.com/sanflg/SeleniumTest

Krishnan Mahadevan
  • 14,121
  • 6
  • 34
  • 66
Santiago Lataza
  • 13
  • 2

0 Answers0