Run each test of in separate VM using JUnit in Ant

Question

I have a Ant build file which uses batchtest of junit to test all the test classes.

Suppose I have a class

class MyTest{
    @BeforeClass
    public static void setUpBeforeClass(){
       //some stuff
    }

    @Test
    public void test1(){
    }

    @Test
    public void test2(){
    }
}

I know that JUnit creates a new instance of MyTest class for each test method, but what I want is that a new VM should be created for each test method. I want each test method to rum in a separate VM and want classloader to load MyTest again for each test method. Is that possible?

I have tried to read the documentation and tried this solution:

<junit fork="yes" reloading="true" forkmode="perTest">
    <batchtest>
    </batchtest>
</junit>

But even after using these options for each of my test method the setUpBeforeClass method is called only once. Am I doing something wrong?

EDIT:

Why do I want to do that?

My test methods are using some collaborators that use static stuff and I want that static stuff to be cleared on every method. Actually I am facing an issue in which tests pass at local environment and fail at production.

Any particular reason for this? Just thinking if there could be a work-around depending on what you need the new VM requirement for... — anishthecoder, Oct 11 '13 at 07:34
@anishthecoder My test methods are using some collaborators that use static stuff and I want that static stuff to be cleared on every method. Actually I am facing an issue in which tests pass at local environment and fail at production. — Narendra Pathai, Oct 11 '13 at 07:35
See if this helps [Java: how to “restart” a static class?](http://stackoverflow.com/questions/4631565/java-how-to-restart-a-static-class) — anishthecoder, Oct 11 '13 at 07:38
@anishthecoder Thanks. Yes I know I can do that, but the problem is I am not sure where the static stuff relies but I think it is because of that as tests pass at local environment and only fail during build. — Narendra Pathai, Oct 11 '13 at 07:42
Hmnn ok. But even if you start a new VM instance with every test method, isn't it still local...? You'd need something more than just a new VM instance, no? Something that starts up the production environment VM in your local setting or something...? — anishthecoder, Oct 11 '13 at 07:53
Keep in mind that using a new JVM for each test method will slow down test execution considerably. It may be better to fix the tests so that they don't step on each other's toes. — Peter Niederwieser, Oct 12 '13 at 04:33
@PeterNiederwieser yes I am aware of that, but I just wanted to find out if that was really the problem or its something else. Do you know how to do that in ant? — Narendra Pathai, Oct 12 '13 at 06:46

visola · Answer 1 · 2014-01-17T18:43:20.157

The only possible way I can see that working is to have multiple instances of <test> elements.

The problem with that is that you have to specify for each <test> element what methods you want to run. I created a single test class that have a static initializer and two tests in it. With the following <junit> task I was able to do what you want:

<junit fork="yes">
    <classpath>
        <path refid="classpath" />
    </classpath>
    <formatter type="xml"/>
    <test name="TestSimple" methods="testOne" toDir="firstRun" />
    <test name="TestSimple" methods="testTwo" toDir="secondRun"  />
</junit>

From what I get from the build logs, it's clear that they ran in two different JVMs:

test:
    ...
    [junit] Executing 'C:\Program Files\Java\jdk1.7.0_25\jre\bin\java.exe' with arguments:
    [junit] '-classpath'
    ...
    [junit] 'org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner'
    [junit] 'TestSimple'
    [junit] 'methods=testOne'
    ...
    [junit] Executing 'C:\Program Files\Java\jdk1.7.0_25\jre\bin\java.exe' with arguments:
    [junit] '-classpath'
    ...
    [junit] 'org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner'
    [junit] 'TestSimple'
    [junit] 'methods=testTwo'
    ...
    [junit] Test TestSimple FAILED

I also see the System.out statement that I put in the static initializer in each test's report.

The important detail here is that you have to make each <test> tag redirect their report to a different directory, using the attribute toDir or change the name of the output file using the attribute outfile. That's because the report file name uses the name of the class and not the method you're running. Which means the second run will overwrite the first run's report.

In your specific case, you could use a <batchtest> to all tests that can run in the same JVM without problems and add an <exclude> tag to avoid running the problematic tests. Then add a specific <test> tag for each problematic test.

They do that because the methods attribute is specifically designed to re-run specific methods or separate tests that are running too slow. From the documentation:

The methods attribute can be useful in the following scenarios:

A test method has failed and you want to re-run the test method to test a fix or re-run the test under the Java debugger without having to wait for the other (possibly long running) test methods to complete.

One or more test methods are running slower than expected and you want to re-run them under a Java profiler (without the overhead of running the profiler whilst other test methods are being executed).

But personally, and coming from a person that ran into the same problem as you before, static initializers suck! I would try really hard to get rid of them as soon as possible. And to quote @PeterNiederwieser, having a new JVM for each set of tests will make running your test suite very slow!

Run each test of in separate VM using JUnit in Ant

1 Answers1

Linked