1

So here's my situation: I have a test that I would run multiple times and I'd like to set some sort of error tolerance i.e. if one run fails it doesn't report the testcase to have Failed. Say I run it 10 times and my target is that it should work properly at least 8 times.

A trivial solution would be to run the test in a loop and use a success count to determine whether to pass or fail a test case.

I was wondering if there was a better method. Perhaps a Junit rule along with the RunWith annotation ?

Thanks In Advance.

revanthb3000
  • 11
  • 2
  • 5
    If you can afford to use TestNG instead, it has exactly that (`@Test(successPercentage = 80.0)`) – fge Jun 16 '13 at 18:26
  • 2
    I would try to test something which works every time. even if you pass 80% of the time, you have a small chance of failing 10 times in a row. – Peter Lawrey Jun 16 '13 at 18:34
  • @PeterLawrey one in a million kind of small – flup Jun 16 '13 at 19:20
  • @flup true, but you have a good chance of 30% or 40% failure sometimes. If the problem is performance related, you can have a slow machine for all ten tests, even if it has a 20% chance of being slow. One place I consult at the tests start breaking on the build server from Friday lunch until after 5 PM as every one is checking stuff in before the weekend. By 6 PM all the test work again :P – Peter Lawrey Jun 16 '13 at 20:03
  • @fge Sadly TestNG is not a viable option for me right now. (no one in the team uses it) – revanthb3000 Jun 17 '13 at 11:28
  • 1
    @PeterLawrey It could be possible that sometimes tests could fail due to maybe a service being down or there being invalid data in the database (which is open to all). I wouldn't want one lousy failure to cause the whole suite to fail. – revanthb3000 Jun 17 '13 at 11:29
  • This seems like a very similar use case to How to Re-run failed JUnit tests immediately? http://stackoverflow.com/questions/8295100/how-to-re-run-failed-junit-tests-immediately/8301639#8301639 or you can use a very similar solution – Matthew Farwell Jun 17 '13 at 19:01
  • Thanks @MatthewFarwell. That seems relevant to my needs. Extended the BlockJunitTest4ClassRunner and added my own runchild method. Seems to be working as expected. Thanks guys ! – revanthb3000 Jun 19 '13 at 07:08

0 Answers0