1

I read the following links, but no solution found yet:

Would it be possible to add a Java method in the Runner class to combine multiples, such as the following (last few lines specifically)? This would prevent us from coding multiple feature files specific to a user account and adding in conditional logic to prevent running based on account attributes, such as compliance state, region, ... and allow us to execute a single runner from our CI/CD pipeline for a full regression test of the service.

public class EquipmentRunner {

    private static final String serviceName = "EquipmentService";
    private static final Integer cores = Runtime.getRuntime().availableProcessors();

    @Test
    public void testParallel() {
        Results baseAccountResults = Runner
            .path("classpath:net/charter/equipment/test/api/domain/v1/Equipment_GET.feature")
            .tags("@regression", "~@standarduser")
            // .hook(new ExtentReportHook())
            .karateEnv("dev_smb")
            .outputCucumberJson(true)
            .outputHtmlReport(true)
            .outputJunitXml(true)
            .systemProperty("karate.dsRow", "1")
            .parallel(cores);

        Results CoamAccountResults = Runner
            .path("classpath:net/charter/equipment/test/api/domain/v1/Equipment_GET.feature")
            .tags("@modem", "~@standarduser")
            // .hook(new ExtentReportHook())
            .karateEnv("dev_smb")
            .outputCucumberJson(true)
            .outputHtmlReport(true)
            .outputJunitXml(true)
            .systemProperty("karate.dsRow", "8")
            .parallel(cores);

        Results NonCompliantAccountResults = Runner
            .path("classpath:net/charter/equipment/test/api/domain/v1/Equipment_GET.feature")
            .tags("@nonCompliant")
            // .hook(new ExtentReportHook())
            .karateEnv("dev_smb")
            .outputCucumberJson(true)
            .outputHtmlReport(true)
            .outputJunitXml(true)
            .systemProperty("karate.username", "testCompliance")
            .systemProperty("karate.password", "password")
            .parallel(cores);

        // This is where we would like to combine the results
        Results results = baseAccountResults + CoamAccountResults + NonCompliantAccountResults;

        // These would likely
        generateReport(results.getReportDir());
        assertEquals(0, results.getFailCount(), results.getErrorMessages());
    }
mike
  • 383
  • 1
  • 12
  • yes this probably requires changes to karate that to be honest not many teams seem to need or have asked for. the recommendation is that there are plenty of ways for you to manage the run of all your tests within one Runner. I think it will be more productive to discuss what prevents you from doing that – Peter Thomas Apr 04 '23 at 17:05
  • 1
    The issue is due to the complexity of the relationships within our tests. We support 100s-1000s of different equipment models, firmware versions, and different account characteristics (compliant, inactive, delinquent). Trying to create a single scenario outline to account for all the variations, or separate feature/scenarios for these is excessive overhead. As a result, we have created our tests at a base level (equipment type defined by tags). We then have separate accounts built for the account specific characteristics and our most popular models. Looping thru all these is our desire ^ – mike Apr 04 '23 at 18:49
  • > `excessive overhead` - not if you do it right, refer: https://stackoverflow.com/a/60387907/143475 - I suggest you model the kind of variations you want in a sample project and then it can be discussed. I repeat that we haven't had a team request this kind of merge capability - and complex enterprises seem to be doing fine without it – Peter Thomas Apr 05 '23 at 01:43
  • Looping through stuff like the equipment models is fine in a scenario, but then if I want to loop those same tests through different firmware versions and then different account characteristics, I can call from them within a scenario outline of characteristics, but then I lose the details in the summary report as everything is combined under the calling feature so no true pass/fail counts are returned. I can also flatten this data into a single table but even using a small subset of 3 models/firmware versions/account characteristics I would end up with 27 rows. Another item would add 9 rows. – mike Apr 05 '23 at 15:39
  • I honestly don't see a problem with "many rows" if that is indeed the business need. I would question if seeing a "break up" in an HTML report is that important, if your basic testing goals are met". see if this thread gives you any ideas: https://stackoverflow.com/a/61685169/143475 - and Karate supports using a JS function to generate or "explode" scenarios based on simple rules, which be the right solution in this case: https://github.com/karatelabs/karate#json-function-data-source – Peter Thomas Apr 05 '23 at 16:17

0 Answers0