How to test assertions?

Question

I am using a unit testing framework to test my libraries. I have quite a few assertions in the library to make sure that programmer errors are caught in debug builds. Now I want to make sure I am testing for all possible programmer errors.

E.g. in a Table class, I want to make sure that the rows and cols passes are not greater than the rows and cols the table has. Let's assume I forget to test for cols. I would like to have my unit tests perform a test where the assertion should fire, and if not, fail the test. Is that possible?

You should specify what *unit testing framework* you are using, as the answer is dependent on that. — David Rodríguez - dribeas, Jun 12 '11 at 23:40
Cool, -1 without an explanation :) @David: I am using CATCH testing framework. I will go with wheaties suggestion and convert the assertions to exceptions just for the unit-test library and test with a no-throw. — Samaursa, Jun 12 '11 at 23:59

score 2 · Accepted Answer · answered Jun 12 '11 at 23:30

The question then becomes are you planning on refactoring your code so that if this condition occurs you'll throw an exception instead of relying on <cassert> functionality to clue you into a problem? If so, you can just check that the exception was thrown. If not, then it's going to be more difficult to test an assert statement from <cassert>. Unit test frameworks like CUTE have an ASSERT_THROWS macro just for exception testing. I'd check your framework.

Also, it's been the case with the shops where I've worked that they frown upon assert and prefer exceptions. Calling abort doesn't help automated testing. Actually, it prohibits it. Just my two cents.

I am compiling without exceptions, so I can only use assertions. I will look into converting assert macros into assertions just for the unit-test project — Samaursa, Jun 12 '11 at 23:59

score 2 · Answer 2 · answered Jun 12 '11 at 23:44

There are three possible ways:

converting assertions to exceptions, or
running each test a separate program whose exit code you (or rather, the test framework) checks, or
let an assertion that fires, write information about itself, which the test framework then can pick up.

As far as I know no commercial or widely used unit test framework supports the last two ways. I have used checking of process exit code for hobby programming, but only with a small personal unit test framework implemented in Python and C++ ("hobby programming": that means I have no good data on how well it scales to large scale programming). The main case for process exit code testing is where the code has static assertions, that you want to be sure are triggered when they are intended to be.

Summing up, with extant test frameworks, conversion to exception is AFAIK your only option.

Cheers & hth.

Thanks, converting them to exceptions is what I will do (+1) — Samaursa, Jun 13 '11 at 00:02

score 0 · Answer 3 · edited May 23 '17 at 12:29

0

The answer is dependent on the particular test framework that you are using, and will probably be found with very little searching in google.

The first hit for "boost unittest test assert" points to this question in StackOverflow: Testing for assert in the Boost Test framework

The second hit for "cppunit test assert" points to this documentation page: Making Assertions

Try searching in the internet for your concrete framework.

edited May 23 '17 at 12:29

Community

1
1

answered Jun 12 '11 at 23:35

David Rodríguez - dribeas

204,818
23
294
489

I tried relying on SO's search but I guess that was a bad idea (a lot of times one enters incorrect search terms as well) – Samaursa Jun 12 '11 at 23:58

How to test assertions?

3 Answers3