6

Is it possible to use the C++ CATCH framework to verify that an assert statement correctly identifies an invalid precondition?

// Source code
void loadDataFile(FILE* input) {
  assert(input != NULL);
  ...
}

// Test code
TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   loadDataFile(NULL)
   // Now what do I look for?
}
Zack
  • 6,232
  • 8
  • 38
  • 68
  • 1
    I don't know for catch, but google test has so called _death tests_. – πάντα ῥεῖ Jul 22 '16 at 18:48
  • 1
    Can you roll your own `assert` macro? You could give it a customizable "assert handler" which throws on failure in your test projects, and terminates otherwise. Then in your tests you could check for `AssertException` thrown. – KABoissonneault Jul 22 '16 at 18:56
  • Please check my answer here: https://stackoverflow.com/questions/37499284/how-to-suppress-termination-in-google-test-when-assert-unexpectedly-triggers/37503591#37503591 –  Jul 24 '16 at 11:33

3 Answers3

3

You may be interested in the google test framework. It has the ability to catch abnormal program termination with:

ASSERT_DEATH(statement, regex);
ASSERT_DEATH_IF_SUPPORTED(statement, regex);
ASSERT_EXIT(statement, predicate, regex);

EXPECT_DEATH(statement, regex);
EXPECT_DEATH_IF_SUPPORTED(statement, regex);
EXPECT_EXIT(statement, predicate, regex);

regex matches text on stderr predicate matches the program's exit code.

I suspect this works by forking the test program before the assertion.

documentation here:

https://github.com/google/googletest/blob/master/googletest/docs/advanced.md

rold2007
  • 1,297
  • 1
  • 12
  • 25
Richard Hodges
  • 68,278
  • 7
  • 90
  • 142
3

Assuming that the first section of your example is the source code under test and the second part is the unit test, then you'll need to make a choice in how you handle this:

Some open-source frameworks like BDE and Boost have their own ASSERT macro which can be configured at application startup to behave different than C assert. For example, you can specify that a failed ASSERT throws an exception - and then you could use Catch's REQUIRE_THROWS() assertion to verify that your code enforced it's non-NULL FILE descriptor contract.

BDE example

#include <bsls_assert.h>

void loadDataFile(FILE* input) {
  BSLS_ASSERT_OPT(input != NULL);
  ...
}

TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   // Opt-in to the 'throw exception on assert failure' handler
   // just for this test case.
   bsls::AssertFailureHandlerGuard guard(&bsls::Assert::failThrow);
   REQUIRE_THROWS_AS(loadDataFile(NULL), bsls::AssertFailedException);
}

Boost example

#include <boost/assert.hpp>

void loadDataFile(FILE* input) {
  BOOST_ASSERT(input != NULL);
  ...
}

namespace boost {
void assertion_failed(char const * expr, char const * function, char const * file, long line) {
    throw std::runtime_error("Assertion Failed"); // TODO: use expr, function, file, line
}
}

TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
   REQUIRE_THROWS(loadDataFile(NULL));
   // Now what do I look for?
}

You could roll your own assert() macro. This is re-inventing the wheel - see above examples instead.

You could change your code to throw std::invalid_argument() exception instead:

  void loadDataFile(FILE* input) {
    if (input == NULL) {
      throw std::invalid_argument("input file descriptor cannot be NULL");
    }
    ...
  }

You can test that your code enforces it's contract with:

REQUIRE_THROWS_AS(loadDataFile(NULL), std::invalid_argument);

This is introducing exceptions (and the need to handle them) into your code and this is a bigger change than you clients may be happy with - some companies have a no-exceptions rule, some platforms (e.g. embedded) do not support exceptions.

Finally, if you really wanted to, you could alter your code's interface to expose contract failure:

enum LoadDataFile_Result {
  LDF_Success,
  LDF_InputIsNull,
  ...
};

LoadDataFile_Result loadDataFile(FILE* input) {
  if (input == NULL) {
    // bail out early for contract failure
    return LDF_InputIsNull;
  }
  // input is non-NULL
  ...
  return LDF_Success;
}

...but this has the inherent risk that clients don't check the return value, the cause of many bugs, and feels like C all over again.

JBRWilkinson
  • 4,821
  • 1
  • 24
  • 36
  • 1
    This doesn't work for asserts used in noexcept functions — specifically, destructors. – Rusty Shackleford May 09 '18 at 18:08
  • @RustyShackleford that’s right, that’s why it is so hard to decide to put noexcept in a function. If you put noexcept in a function you cannot test the failure of the function. It is also not (automatically) testable. Inverting the logic you should only put noexcept in functions that cannot (under any current or future implementation) possibly fail. That is, noexcept should be used only for functions with a “wide contract”. Look for “Lakos rule”. This is why so few functions are noexcept in the standard for example. – alfC Dec 02 '20 at 08:43
1

If you can mock C functions, you could temporarily change the assertion failure to an exception.

For example, using Hippomocks your test case would look something like:

#ifndef NDEBUG
TEST_CASE("loadDataFile asserts out when passed NULL", "[loadDataFile]") {
    MockRepository mocks;
    mocks.OnCallFunc(__assert_fail).Throw(nullptr);

    CHECK_THROWS_AS(loadDataFile(NULL), std::nullptr_t);
}
#endif

Portability may be an issue, I've only tested this on Linux with glibc, and musl.

Phil
  • 353
  • 4
  • 7