Narrowing down a bad test by running junit tests in various combinations using Maven or other tool

Question

I have 1000 junit tests and one "bad" test that is modifying a shared resource causing a subsequent test to fail. It passes if run alone. I'm looking for a Maven plugin or Java application or tool that will take as input a test class name. It will then run the 1000 tests in various combinations until it finds the "bad" test. Assume it is one "bad" test.

Answer 1

As you may noticed, writing dependant tests IS the problem ! Every solutions would be a workaround, maybe with side effectfs, complexity the first one.

Assuming you REALLY can't change that, you may have different strategies to solve your issue, but they are not necessary related to maven :

Ordering

Order your test to run the "bad one" at the end ... be carefull that you me affected again if you have a second bad test !

1. User TestNG instead of JUnit and @Groups and @AfterGroups annotations to split your tests and run them as you want
2. Use @AfterClass and BeforeClass with a test suite: Cleanup after all junit tests
3. Manually describe a Test Suite (not sure if you can achieve what you want)

Provide good data

1. Use setUp and tearDown methods to prepare and clean data in order to always have a stable environment, on each tests classes
2. Rollback the test that modify you resource (pretty the same thing indeed)

Step back thoughts :

Just a piece of minds :

If you cannot run tests independently, they are not unit tests.