10

My setup:

  • A TestBase class containing a @BeforeClass method
  • Several Test classes extending from TestBase class and also containing a @BeforeClass method
  • testNG 6.8.8

Why this setup?:

  • I need the @BeforeClass in the TestBase class to provide setup that all testclasses will need and I don't want to repeat in every Test class. For example thread-id-dependent login credentials.
  • TestBase class also instantiates the Selenium WebDriver
  • I need the @BeforeClass in the Test classes to initialize everything that all @Test methods will need to use but that only needs to (or must) be built/invoked once for all tests. This includes calls to said WebDriver instance (that's why a "normal" constructor doesn't work here)

Here's what happens:

When I run the tests via a testNG xml file and there is an exception within the @BeforeClass method of one of the Test classes, then all subsequent Test classes are skipped by TestNG.

Why does this happen? How to prevent it?

When I change the annotation in the TestBase class to @BeforeSuite for example, then all tests are run, even if there is an exception in on of the @BeforeClass methods.

Example:

When you run the xml file, complete RunAllTestClasses02 class is skipped.

testNG xml file:

<?xml version="1.0" encoding="UTF-8"?>

<suite name = "MiscSuite">
    <test name = "MiscTest">
        <classes >
            <class name="drkthng.misc.RunAllTestClasses01" />
            <class name="drkthng.misc.RunAllTestClasses02" />
        </classes>
    </test>
</suite>

TestBase class with a @BeforeClass method:

public abstract class RunAllTestClassesBase {

    @BeforeClass
    public void beforeClass() {
        // do something that all Test classes will need
    }
}

Test class that throws Exception within @BeforeClass method:

public class RunAllTestClasses01 extends RunAllTestClassesBase {

    @BeforeClass
    public void beforeClass() {
        Assert.assertTrue(false);
    }

    @Test
    public void Test01() {
        Assert.assertTrue(true);
    }
}
drkthng
  • 6,651
  • 7
  • 33
  • 53
  • maybe [this](http://stackoverflow.com/questions/2132734/beforeclass-and-inheritance-order-of-execution) answer could help – Erki M. Jul 18 '15 at 03:48
  • @ErkiM. thanks man! this definitely sheds some light. Still there's the question why all subsequent Test are skipped, just because one! class fails. – drkthng Jul 18 '15 at 10:25
  • I believe it's by design as your tests depend on the outcome of `@BeforeClass` and it fails, hence the execution is stopped. To over come this, you could annotate your tests with `alwaysRun`, that by the [docs](http://testng.org/doc/documentation-main.html) : `alwaysRun If set to true, this test method will always be run even if it depends on a method that failed.` I have not tested this myself, let me know how it works out for you. – Erki M. Jul 18 '15 at 13:45
  • 1
    Bug in testng I suppose - fixed in 6.9.5 it seems : https://github.com/cbeust/testng/issues/471 – niharika_neo Jul 19 '15 at 05:03
  • @niharika_neo thank you so much!!! this solved the whole issue!! won't you put this as an answer? because that's what it is regarding my updated question (testng version number 6.8.8) - otherwise I'll have to do this myself und you won't get the whole credit ;-) – drkthng Jul 19 '15 at 13:31
  • :) done! Now give me my points! :D – niharika_neo Jul 20 '15 at 13:06

2 Answers2

4

This was a bug in Testng. solved in 6.9.5. Please upgrade.

niharika_neo
  • 8,441
  • 1
  • 19
  • 31
  • 4
    unfortunately the bug is not completely resolved -> Problem with children being skipped when one child fails during its own \@beforeClass method still holds, if children have \@beforeClass and the parent class has \@beforeClass AND \@afterClass. – drkthng Jul 27 '15 at 09:05
  • 1
    Same here - upgrading didn't solve it for me. Switching to `@BeforeSuite` and `@AfterSuite` did. – Nate Barbettini Apr 01 '16 at 18:39
  • 4
    still see the bug even on 6.11 – danggrianto Oct 17 '17 at 16:14
0

Try to add @AfterClass(alwaysrun = true) or/and @AfterMethod(alwaysrun=true) as by default they are skipped if either BeforeClass or BeforeMethod are not completed.

The documentation on testNG Configuration Failures, Policy, and alwaysRun explains whether/when configuration failures cause listener methods (alwaysRun and other listeners) to be skipped, failure policies and best practices.

DaveFar
  • 7,078
  • 4
  • 50
  • 90
Dexter
  • 98
  • 7