8

Is maven-surefire-plugin able to run all JUnit test methods in isolation? I.e., is it able to fork a JVM for each test method rather than for each test class?

The deprecated option

<forkMode>pertest</forkMode>

and the current

<forkCount>1</forkCount>
<reuseForks>false</reuseForks>

seem to only fork for each test class.

PS: Test methods should be independent and therefore no one should need to run each one on a new JVM (and not to say that it would be very expensive). But I was wondering whether there is such option or not.

josecampos
  • 851
  • 1
  • 8
  • 25
  • [IntelliJ seems to be able to do it.](https://stackoverflow.com/a/54225019/4179032) I'm waiting for a Surefire solution too. – Leponzo Jan 08 '21 at 04:26

2 Answers2

1

The Maven Surefire Plugin supports this configuration:

<parallel>methods</parallel>

For JUnit versions >= 4.7

For example:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.20</version>
    <configuration>
      <parallel>methods</parallel>
      <threadCount>10</threadCount>
    </configuration>
  </plugin>

More details here.

Edit 1 based on your comment:

I would like to run each test method on a new JVM, and not on another thread on the same JVM

When using reuseForks=true and a forkCount > 1, test classes are handed over to the forked process one-by-one. So, with parallel=methods classes are executed in forkCount concurrent processes, each of these processes uses a threadCount number threads to execute the methods of one class in parallel.

So, it looks like surefire does not support your use case.

FWIW, spawning a JVM for every test method sounds like it could get expensive. Are you perhaps dealing with an issue where test methods have unwanted side effects within a JVM (maybe they set System properties or change statics) and these side effects cannot be isolated? If so, perhaps revisting the tests to eliminate these side effects might be more desireable than implementing a bespoke and potentially expensive test runner?

glytching
  • 44,936
  • 9
  • 114
  • 120
  • Thanks @glitch. According to the link you pointed out, "The important thing to remember with the parallel option is: the concurrency happens within the same JVM process.". So, by defining 1false I can get a new JVM per test class, but then if I also include methods10 I only get 10 test methods executing concurrently on the **same JVM**, and not each test method on a new JVM. Right? – josecampos Aug 31 '17 at 10:08
  • Yes, the `classesAndMethods` indicates your intent for parallelisation and the `forkCount` limits the extend of the concurrency. – glytching Aug 31 '17 at 10:10
  • Thanks, but that it isn't what I'm looking for. I would like to run each test method on a new JVM, and not on another thread on the same JVM. – josecampos Aug 31 '17 at 10:21
  • @josecampos I've updated my answer following this clarification: "I would like to run each test method on a new JVM, and not on another thread on the same JVM" – glytching Aug 31 '17 at 10:32
  • I totally agree with your "Edit 1": running each test method on a new JVM would be insanely expensive, and as you said there could also be some unwanted side effects of doing that. And I also agree if test methods do actually need to be executed isolation (i.e., on a new JVM) they most likely need to be re-written. I was just wondering whether there is an answer (i.e., a sufire option) to my question, not that I actually need or want to implement such thing. – josecampos Aug 31 '17 at 10:43
  • Ok, well then the answer is **no** :) – glytching Aug 31 '17 at 10:44
0

I found this project which claims to do it, though it did not seem to work when I tried (maybe out of date). Anyway I imagine this could not work for anything but basic @Test-annotated methods: no @RunWith(Parameterized.class) etc.

Jesse Glick
  • 24,539
  • 10
  • 90
  • 112