How to tell Boost.Test to stop on first failing test case?

Question

I've got a number of Boost test cases ordered in several test suites. Some test cases have one, some more than one check.

However, when executing all tests, they all get executed – no matter how many fail or pass. I know, that I can stop the execution of one test case with several checks by using BOOST_REQUIRE instead of BOOST_CHECK. But that's not want I want.

How can I tell Boost to stop the whole execution after the first test case failed? I would prefer a compiled solution (e.g. realized with a global fixture) over a runtime solution (i.e. runtime parameters).

BOOST_REQUIRE_THROW will throw an exception so execution of the entire test suite should be stopped. — TemplateRex, Apr 24 '12 at 15:00
Thanks @rhalbersma. I'll try that. Why don't you make an answer out of that? — Torbjörn, Apr 25 '12 at 08:12
Ehm, because after more careful reading of the Boost.Test manual, the suggestion was inaccurate. BOOST_REQUIRE_THROW is a unit test to check whether an exception is thrown. It doesn't throw itself. — TemplateRex, Apr 25 '12 at 08:17

score 8 · Accepted Answer · answered May 08 '12 at 12:50

BOOST_REQUIRE will stop the current test case in a test suite but go on on others.

I don't really see what you wanted when you asked for a "compiled solution" but here is a trick that should work. I use a boolean to check the stability of the whole test suite. If it is unstable i.e a BOOST_REQUIRE has been triggered then I stop the whole thing.

Hope it can help you.

//#include <...>

//FIXTURES ZONE
struct fixture
{
    fixture():x(0.0),y(0.0){}
    double x;
    double y;
};

//HELPERS ZONE
static bool test_suite_stable = true;

void in_strategy(bool & stable)
{
    if(stable)
        {
            stable = false;
        }
    else
        {
            exit();
        }
}

void out_strategy(bool & stable)
{
    if(!stable)
        {
            stable = true;
        }
}

BOOST_AUTO_TEST_SUITE(my_test_suite)

//TEST CASES ZONE
BOOST_FIXTURE_TEST_CASE(my_test_case, fixture)
{
    in_strategy(test_suite_stable);
    //...
    //BOOST_REQUIRE() -> triggered
    out_strategy(test_suite_stable);
}

BOOST_FIXTURE_TEST_CASE(another_test_case, fixture)
{
    in_strategy(test_suite_stable); //-> exit() since last triggered so stable = false
    //...
    //BOOST_REQUIRE()
    out_strategy(test_suite_stable);
}

BOOST_TEST_SUITE_END()

Benoit.

Thanks for this hack :) But calling in_/out_strategy in each test case is a little towards the DRY principle. Isn't there the possibility to globally (or per test suite) define `before` and `after` fixtures run for every test case? Alternatively one has to redefine BOOST_AUTO_TEST_CASE or alike. — Torbjörn, May 09 '12 at 08:02
@Torbjoern - put the in/out strategy call in the ctor/dtor of the fixture. The fixture is created and destructed for *each* test case separately. Quanteek - might want to include that in your answer? — Martin Ba, Nov 03 '14 at 21:39

score 1 · Answer 2 · answered May 24 '12 at 22:24

1

Why not just use assert? Not only you abort immediately the whole program, you will also be able to see stack if necessary.

answered May 24 '12 at 22:24

Gennadiy Rozental

1,905
2
13
17

How to tell Boost.Test to stop on first failing test case?

2 Answers2