0

I'm trying to run 2 Cucumber tests in parallel and sequential using TestNG and SpringBootTest but when my tests execute the following happens

mvn test

2 browsers open and both navigate to the Wikipedia homepage.

if you add 2 more scenarios it opens those many threads per scenario, I don't have any control over the number of threads to execute.

How to control the number of threads and dataprovider count, any help is appreciated.

Repo : https://github.com/cmccarthyIrl/spring-cucumber-testng-parallel-test-harness

automatenew_jog
  • 1
  • 2

2 Answers2

0

The possible reason is, the runner you are using converts it into testng data driven test with single test with scenarios from each feature file supplied through data-provider. This is not a right approach. However, in testng there is separate property to set thread count for data driven test. You can set data-provider-thread-count in xml configuration file at suite lever or can pass command-line argument -dataproviderthreadcount to specify number of threads.

Better approach

You can look into another library qaf-cucumber with native testng implementation. It is considering each scenario as testng test method gives more control and utilization of each feature of testng. With this library, only scenario with examples are converted as testng data driven test.

You don't need to have additional class to run test. Just use factory available class to have different configuration combinations. Here is sample configuration file:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="QAF Demo" verbose="1">
    <test name="Web-Suite"  enabled="true">
        <classes>
            <class name="com.qmetry.qaf.automation.cucumber.runner.CucumberScenarioFactory" />
        </classes>
    </test>
</suite>

Note: As of today qaf-cucumber supports cucumber 5.x

user861594
  • 5,733
  • 3
  • 29
  • 45
  • It's worth pointing out that QAF can only do this because they're overriding the internals of both TestNG and Cucumber. This means that you are effectively using neither TestNG nor Cucumber. And as such you are wholly dependent on QMetry to keep their project up to date. Currently they're somewhat behind on that. – M.P. Korstanje Jul 30 '22 at 00:48
  • @M.P.Korstanje, it's other way around. With QAF you are effectively using both TestNG and Cucumber. Refer [CucumberScenarioFactory](https://github.com/qmetry/qaf-cucumber/blob/9f4b5e90d56e93c01aa2d8a018c4d39ace7b0b1f/src/com/qmetry/qaf/automation/cucumber/runner/CucumberScenarioFactory.java) which creates [CucumberScenario](https://github.com/qmetry/qaf-cucumber/blob/9f4b5e90d56e93c01aa2d8a018c4d39ace7b0b1f/src/com/qmetry/qaf/automation/cucumber/runner/CucumberScenario.java) in order to follow lifecycle of both cucumber and testng. – user861594 Jul 31 '22 at 02:10
  • To do what QAF does, it has had to replace parts of Cucumber and TestNGs internals. For example [QAF shadows these parts of TestNG](https://github.com/qmetry/qaf/tree/master/src/org/testng). This allows it to do things that aren't possible with an unmodified TestNG. But it also means that it is also no longer TestNG, but rather a fork provided by QMetry. From a software perspective this is bad because it means you are wholely dependent on QMetry to stay up todate (which they don't). – M.P. Korstanje Jul 31 '22 at 10:01
  • For what it is worth, using a `@Factory` annotation to generate test cases for each scenario is a nice solution. Unfortunately without modifying TestNG the reporting tools will primarily render the method name rather then the instance name provider by implementing `ITest` (i.e. the scenario name). If the reporting tools were better I would have added this to Cucumber already. – M.P. Korstanje Jul 31 '22 at 10:08
  • @M.P.Korstanje regarding what you seeing shadow classes from testng is only to continue support older version of TestNG. Lots of enterprise users still uses old version of TestNG. Regarding reporting QAF-cucumber supports all cucumber report plugins as is and yes with scenario name in report!. One can pass cucumber options, it allows to provide all supported cucumber options using property with `cucumber` prefix. – user861594 Jul 31 '22 at 18:24
  • I'm talking about the native TestNG reports for which you claimed QAF provided native support. – M.P. Korstanje Jul 31 '22 at 19:41
  • And I'm skeptical about your claim that this shadowing is needed to support older versions. For example QAFs https://github.com/qmetry/qaf/blob/master/src/org/testng/internal/NoOpTestClass.java is shadowing a class in the current version https://github.com/cbeust/testng/blob/master/testng-core/src/main/java/org/testng/internal/NoOpTestClass.java. – M.P. Korstanje Jul 31 '22 at 19:45
  • Anyway, I see that the main point goes unchallenged. I'll leave it at that. – M.P. Korstanje Jul 31 '22 at 19:46
  • Native because, its considering each scenario as testng test and follows complete testng lifecycle, at the same time follows cucumber scenario/feature execution lifecycle. Regarding report, it generates testng reports with correct test name (scenario name) and [qaf reporting](https://qmetry.github.io/qaf/latest/qaf_reporting.html) in addition to cucumber report . Regarding `NoOpTestClass` class, i see the comment "Added alternate to convert testng method to scenorio at early stage through `NoOpTestClass` **to fix issues while running from ide as testng test**". – user861594 Aug 01 '22 at 17:09
  • You can try qaf-cucumber once and i am sure you will like it and will start contributing to it to support latest cucumber version :). – user861594 Aug 01 '22 at 17:09
  • That's a bit rich coming from a company that didn't contribute back. – M.P. Korstanje Aug 01 '22 at 19:15
0

TestNG runs dataproviders with a parallism of 10 by default. You can tell Maven to tell TestNG to use fewer threads.

https://github.com/cucumber/cucumber-jvm/tree/main/testng#parallel-execution

<plugins>
   <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
         <properties>
            <property>
               <name>dataproviderthreadcount</name>
               <value>${threadcount}</value>
            </property>
         </properties>
      </configuration>
   </plugin>
</plugins>
M.P. Korstanje
  • 10,426
  • 3
  • 36
  • 58
  • it tried with sureFire-plugin and all the tests got skipped. You can give a try Repo : https://github.com/cmccarthyIrl/spring-cucumber-testng-parallel-test-harness – automatenew_jog Jul 29 '22 at 03:31
  • Then you shouldn't worry yet about tests running in parallel. You're currently using Spring which provides JUnit 4 or 5 depending on version. So Surefire will default to those instead of TestNG. See: https://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html#running-testng-and-junit-tests to run multiple test frameworks together. Or instead of using TestNG use the latest Spring Boot and Cucumbers JUnit 5 integration https://github.com/cucumber/cucumber-jvm/tree/main/junit-platform-engine – M.P. Korstanje Jul 29 '22 at 08:43