5

I have a simple function which I want to test. The function goes like this:

void func()
{
    // do some work
    ...
    if(error_detected)
    {
        fatal_error("failure...");
        exit(1);
    }
}

Now I have to write a test which generates an error. Only the exit(1) exists the test with a failure, none the less!

How is this case usually handled?

I can rewrite/change the function code since I'm in full control of the entire project. However, I'm using cppunit and am hoping I can have that as one of the test in the suite.

Update:

One note that I'd like to make in regard to some of the solutions proposed: Added an interface that can be implemented differently depending on whether we are testing or not is not as strong a way to test as it may look like. Why is that? Because I would be testing with an interface which implementation is different in the test than in the real world. The result of the test does not prove that the real world situation would work right. At least not 100% (it only proves that the path is indeed taken in that particular situation.)

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156

4 Answers4

4

Normally, you should always be able to decide from the client code how your errors are handled.

Edit: consider a scenario where you write a parsing library, and a HTTP server project wants to use it. While your library may have extraordinary features, they will not be able to use it, because every time the server provides invalid inputs to your library, instead of allowing the server to report a HTTP 4XX (invalid request) error to the client, your library shuts down the whole server indiscriminately (i.e. your library calls exit() internally).

If possible in your situation, throw an exception:

void func()
{
    // do some work
    ...
    if(error_detected)
        throw std::runtime_error("failure..."); // no exit call at all
}

Production code (decide here if calling exit):

try { func(); } catch(const std::runtime_error&) { exit(1); }

Test code (no exit here):

try {
    func();
    // mark test as failed here (should not be reached)
} catch(const std::runtime_error&) {
    // mark test as passed here
}

If you cannot throw an exception in all cases (e.g. your code will be used from C code and should not throw exceptions), you can inject the error reporting code into your function:

typedef void (*on_error)(int error_code);
void func(on_error callback)
{
    // do some work
    ...
    if(error_detected)
        callback(1);
}

Production code:

void exit_on_error(int code) { exit(code); }
func(exit_on_error);

Test code:

enum { no_error = 0 };
int failed = no_error;
void on_error(int code) { failed = code; }

// test code:
func(on_error);
// check value of failed here
utnapistim
  • 26,809
  • 3
  • 46
  • 82
  • Yes. This was code from a compiler and thus not a library. So `exit(1)` made sense if I encounter a *rather fatal error.* But now that I'm (indeed!) converting it to a library that will be used in a web server (as you mentioned!) it would be wise for me to go through the code and change all the `exit(1)` with `throw exception_exit(1, "some message")`. Then the test with cppunit becomes easy since they offer a macro: `CPPUNIT_ASSERT_THROW(test-code, exception_exit)`. That being said, I wanted to see a real good solution with an actual `exit()` which the command line compile will have! – Alexis Wilke Jun 03 '14 at 09:53
  • If you want an example with a library that exits your program, consider using `std::terminate` instead of `exit`; `std::terminate` supports setting a [terminate handler](http://en.cppreference.com/w/cpp/error/terminate_handler). – utnapistim Jun 03 '14 at 13:24
  • `std::terminate` sounds cleaner than a callback as it does not change the behavior of the code being tested. Although the default behavior of std::terminate() is to abort and we cannot pass an exit code... – Alexis Wilke Jun 03 '14 at 20:44
3

If you are using gtest, it has a whole section dedicated to it (see Death Tests), where they say :

any test that checks that a program terminates (except by throwing an exception) in an expected fashion is also a death test.

If you are using google test, then you can test for death test using ASSERT_DEATH.

Otherwise, you need to spawn a process, execute a test, and check the result (return value of the spawned process).

BЈовић
  • 62,405
  • 41
  • 173
  • 273
  • It looks like the Death Tests from that GoogleTest library does just that: spawn a process and verify the output. I was thinking of having to write a separate test, but spawning would work too! – Alexis Wilke Jun 03 '14 at 09:47
3

If the code bugging you is the fact that it calls exit(), abstract it out.

void func(IApplicationServices& app) {
   // ...
   app.exit();
   // ...
}

Then, when you test, pass a mock object instead of a full-fledged object that exits the application. The OS should be considered an external system, abstracted as such, and dependencies injected.

But code throwing an exception would be a cleaner design, because your processing should not know that is it run in a process over which client code has control.

Laurent LA RIZZA
  • 2,905
  • 1
  • 23
  • 41
  • I thought of that approach, but the problem is that some of these exit() can be really deep and thus you'd have to carry this IApplicationServices everywhere... I guess throwing is indeed better. – Alexis Wilke Jun 03 '14 at 09:46
  • Not really. If you're trying to unit test your function, then the real application services could be created by the direct caller, which makes the dependency on the OS bubble up one level. It could even be a member of the containing object.If you're instead writing mini integration tests on legacy code, then you'd have to bubble up a couple of levels, but still create the real app services in the direct caller of the chunk you're testing. Anyway, if you have calls to a system function this deep, you'll have bigger chunks to bubble up to the periphery of your application to have a clean design. – Laurent LA RIZZA Jun 03 '14 at 09:54
2

You can superimpose exit function, probably for the duration of that particular test. See Wrapping Linux shared library functions.

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271
  • Wow! That's a neat solution. A bit scary in a way to think that you can do such things since that means you can wrap really anything for any tool! – Alexis Wilke Jun 03 '14 at 09:49
  • @AlexisWilke Yes, sounds pretty neat. Until you have to implement it for a complex project. Then it becomes pain in the ass – BЈовић Jun 03 '14 at 10:07