11

I have a set of 44 JUnit tests that I run using Eclipse (I got those tests from someone else - I am new to JUnit tests). When I run them all together, 24 of them fail. However, if I then run the failing ones individually, some of them do pass after all. The tests do take a little time - one of the failing ones for example takes about one or two minutes to complete, while just letting all of them run finishes in just a few seconds.

I am starting multiple tests by right-clicking on the folder they are in and selecting "Run As -> JUnit Test". I am using JUnit 3. Am I doing something wrong in starting them / is there some kind of option I am missing?

pyvi
  • 675
  • 1
  • 4
  • 15

3 Answers3

16

It's hard to say for sure without seeing the tests, but it sounds to me like they share some state or resource that is not being reset correctly for the next test.

GaryF
  • 23,950
  • 10
  • 60
  • 73
  • In my case it was private static variable which was changing. I had to reset the variable after every test. Used @After annotation to tag the cleanup method. – ZakiMak Jun 17 '15 at 19:10
5

GaryF's answer is one possibility. Another is that the tests have a race-condition: whether or not a test succeeds depends on how fast something occurs (which can vary due to the vagaries of the O/S). If you run the failing tests separately, do they always succeed or do they sometimes fail. If they sometimes fail, then you likely have a race-condition.

Steve Emmerson
  • 7,702
  • 5
  • 33
  • 59
  • I can't check now because the code is at work, but the 4 or 5 times I tried, they always succeeded. I also did not se anything in the JUnite code to suggest to me that there is a time condition in them. – pyvi Nov 26 '10 at 19:24
  • @pyvi Then GaryF's scenario is more consistent with the evidence. – Steve Emmerson Nov 27 '10 at 00:33
5

To expand on Gary's answer, when right-clicking and doing the Run As -> JUnit, you can't guarantee the order in which the tests are run, which could also help to corrupt a shared resource.

I would start by looking at your setup() and teardown() methods to ensure that shared resources are being reset properly. Also, since you inherited these tests, you may want to look at whether any of the tests are dependent upon one another. While this is a bad practice and should be changed, you could perhaps create a test suite() to ensure the order in which they're run (at least until you can re-factor and decouple the tests).

Riggy
  • 1,347
  • 1
  • 14
  • 26
  • In my case, it was a ehcache misconfiguration. Too localized to be an answer, so commenting the "best fit" answer. See http://stackoverflow.com/a/21458679/837154 – orique Oct 31 '14 at 15:28