Executing Feature in isolation, but contained Scenarios in parallel

Question

I have a large-ish and rapidly growing body of karate tests and use the parallel execution to speed up processing, which basically works great in different configurations:

• Normal parallel execution (vast majority of tests)
• Sequential execution of Scenarios within a Feature (parallel=false) for very few special cases
• Completely sequential execution (via separate single-threaded runner, triggered by custom @sequential tag) for things that modify configuration settings, global lookups etc

There's however also a parameterized (Scenario Outline) feature for the basic functionality of many types of global lookups. Currently it runs in the "completely sequential" mode because it affects other tests. But actually the scenarios inside that feature could be executed in parallel (they don't affect each other) as long as the Feature as a whole is executed in isolation (because the tests do affect other Features).

So - is there a way to implement "sequential Features with parallel Scenarios" execution? I admit that this is likely a niche case, but it would speed up tests execution quite a bit in my case.

Answer 1

... and posting this question already got the ideas flowing and pointed me to a possible way to implement this:

  private static void runLocalParallel(Builder<?> builder) {
    final List<Feature> features = builder.tags("@local_parallel").resolveAll();

    for (Feature feature : features) {
      builder.features(feature).parallel(8);
    }
  }

This identifies all features tagged with @local_parallel, iterates over them and executes a parallel runner for each individually. Result handling, report output etc still needs to be implemented in an elegant manner, but that's doable as well.

Answer 2

Yes, indeed an edge case - but it has come up a few times. We've wondered about a way to "bucketize" threads, which means we can do things like say that certain tags have to be run only on particular thread. Come to think of it, that's a good feature request, so I opened one, feel free to comment. https://github.com/karatelabs/karate/issues/2235

In theory if you write some Java glue code that holds a lock, you can call that code before entering any "critical" feature. I haven't tried it, but may be worth experimenting.