Junit4 run a test class a fixed number of times and display results (eclipse)

Question

I want to be able to run a Test class a specified number of times. The class looks like :

@RunWith(Parameterized.class)
public class TestSmithWaterman {

    private static String[] args;
    private static SmithWaterman sw;
    private Double[][] h;
    private String seq1aligned;

    @Parameters
    public static Collection<Object[]> configs() {
        // h and seq1aligned values 
    }

    public TestSmithWaterman(Double[][] h, String seq1aligned) {
        this.h = h;
        this.seq1aligned = seq1aligned;
    }

    @BeforeClass
    public static void init() {
        // run smith waterman once and for all
    }

    @Test
    @Repeat(value = 20) // does nothing
    // see http://codehowtos.blogspot.gr/2011/04/run-junit-test-repeatedly.html
    public void testCalculateMatrices() {
        assertEquals(h, sw.getH());
    }

    @Test
    public void testAlignSeq1() {
        assertEquals(seq1aligned, sw.getSeq1Aligned());
    }

    // etc
}

Any of the tests above may fail (concurrency bugs - EDIT : the failures provide useful debug info) so I want to be able to run the class multiple times and preferably have the results grouped somehow. Tried the Repeat annotation - but this is test specific (and did not really make it work - see above) and struggled with the RepeatedTest.class, which cannot seem to transfer to Junit 4 - the closest I found on SO is this - but apparently it is Junit3. In Junit4 my suite looks like :

@RunWith(Suite.class)
@SuiteClasses({ TestSmithWaterman.class })
public class AllTests {}

and I see no way to run this multiple times. Parametrized with empty options is not an option really - as I need my params anyway

So I am stuck hitting Control + F11 in eclipse again and again

Help

EDIT (2017.01.25): someone went ahead and flagged this as duplicate of the question whose accepted answer I explicitly say does not apply here

If your test might fail, there is either something wrong with your test or your code. — André Stannek, Jan 19 '13 at 20:56
@Andr:Please comment _on the question_ - I know this (obviously!) I am presenting a use case why I need multiple runs - my test provide debugging info — Mr_and_Mrs_D, Jan 19 '13 at 20:58
Then I got your question wrong. The question itself doesn't present a valid use case to me, that's why I wanted to talk you out of it ;-) — André Stannek, Jan 19 '13 at 21:04
Have you seen my answer to How to Re-run failed JUnit tests immediately? http://stackoverflow.com/questions/8295100/how-to-re-run-failed-junit-tests-immediately/8301639#8301639 — Matthew Farwell, Jan 19 '13 at 21:10
@MatthewFarwell: hmm - right direction :) - have to look it up - this is per Test again (not per TestClass) ? — Mr_and_Mrs_D, Jan 19 '13 at 21:14
This is per test, but there is such a thing as a ClassRule. This does the same thing as a Rule, but at class level. — Matthew Farwell, Jan 19 '13 at 21:15
@MatthewFarwell: a ha - please feel free to adapt your answer here :) I guess this is run via the Junit runner (I am on eclipse and I 'd rather have something like Run as > Junit Test - so I have the results displayed nice and all) — Mr_and_Mrs_D, Jan 19 '13 at 21:17
Possible duplicate of [Easy way of running the same junit test over and over?](http://stackoverflow.com/questions/1492856/easy-way-of-running-the-same-junit-test-over-and-over) — R. Oosterholt, Jan 25 '17 at 07:54
@R.Oosterholt: have you read the question ? I explicitly link to the accepted answer of the question I "duplicated" — Mr_and_Mrs_D, Jan 25 '17 at 18:34

score 1 · Accepted Answer · edited May 23 '17 at 12:33

As suggested by @MatthewFarwell in the comments I implemented a test rule as per his answer

public static class Retry implements TestRule {

    private final int retryCount;

    public Retry(int retryCount) {
        this.retryCount = retryCount;
    }

    @Override
    public Statement apply(final Statement base,
            final Description description) {
        return new Statement() {

            @Override
            @SuppressWarnings("synthetic-access")
            public void evaluate() throws Throwable {
                Throwable caughtThrowable = null;
                int failuresCount = 0;
                for (int i = 0; i < retryCount; i++) {
                    try {
                        base.evaluate();
                    } catch (Throwable t) {
                        caughtThrowable = t;
                        System.err.println(description.getDisplayName()
                            + ": run " + (i + 1) + " failed:");
                        t.printStackTrace();
                        ++failuresCount;
                    }
                }
                if (caughtThrowable == null) return;
                throw new AssertionError(description.getDisplayName()
                        + ": failures " + failuresCount + " out of "
                        + retryCount + " tries. See last throwable as the cause.", caughtThrowable);
            }
        };
    }
}

as a nested class in my test class - and added

@Rule
public Retry retry = new Retry(69);

before my test methods in the same class.

This indeed does the trick - it does repeat the test 69 times - in the case of some exception a new AssertionError, with an individual message containing some statistics plus the original Throwable as a cause, gets thrown. So the statistics will be also visible in the jUnit view of Eclipse.

The AssertionError was edited in by @mineralf - I have not tested it myself but it should be as stated in my edited answer. — Mr_and_Mrs_D, Dec 16 '15 at 17:30

Junit4 run a test class a fixed number of times and display results (eclipse)

1 Answers1

Linked