cmocka malloc testing OOM and gcov

Question

I'm having a hard time finding an answer to a nitch case using cmocka, testing malloc for failure (simulating), and using gcov

Update about cmocka+gcov: I noticed I get empty gcda files as soon as I mock a function in my cmocka tests. Why? Googling cmocka and gcov gives results where people talk about using the two together. It seems most people are using CMake, something I will look at later but there should be no reason (that I can think of) that would require me to use cmake. Why can't I just use cmocka with the --coverage/-lgcov flags?

Orignal question:

I've tried a myriad combinations mostly based off of two main ideas:

I tried using -Wl,--wrap=malloc so calls to malloc are wrapped. From my cmocka tests I attempted to use will_return(__wrap_malloc, (void*)NULL) to simulate a malloc failure. In my wrap function I use mock() to determine if I should return __real_malloc() or NULL. This has the ideal effect, however I found that gcov fails to create gcda files, which is part of the reason with wrapping malloc, so I can test malloc failing AND get code coverage results. I feel I've played dirty games with symbols and messed up malloc() calls called form other compilation units (gcov? cmocka?).

Another way I tried was to us gcc -include using a #define for malloc to call "my malloc" and compile my target code to be tested with mymalloc.c (defining the "my malloc"). So a #define malloc _mymalloc helps me call only the "special malloc" from the target test code leaving malloc alone anywhere else it is called (i.e., leave the other compilation unites alone so they just always call real malloc). However I don't know how to use will_return() and mock() correctly to detect failure cases vs success cases. If I am testing malloc() failing I get what I want, I return NULL from "malloc" based on mock() returning NULL- this is all done in a wrapping function for malloc that is only called in the targeted code. However if I want to return the results of the real malloc than cmocka will fail since I didn't return the result from mock(). I wish I could just have cmocka just dequeue the results from the mock() macro and then not care that I didn't return the results since I need real results from malloc() so the code under test can function correctly.

I feel it should be possible to combine malloc testing, with cmocka and get gcov results.

whatever the answer is I'd like to pull of the following or something similar.

int business_code()
{
    void* d = malloc(somethingCalculated);
    void* e = malloc(somethingElse);
    if(!d) return someRecovery();
    if(!e) return someOtherRecovery();
    return 0;
}

then have cmocka tests like

cmocka_d_fail()
{
    will_return(malloc, NULL);
    int ret = business_code();
    assert_int_equal(ret, ERROR_CODE_D);
}

cmocka_e_fail()
{
    will_return(malloc, __LINE__); // someway to tell wrapped malloc to give me real memory because the code under test needs it
    will_return(malloc, NULL); // I want "d" malloc to succeed but "e" malloc to fail
    int ret = business_code();
    assert_int_equal(ret, ERROR_CODE_E);
}

I get close with some of the #define/wrap ideas I tried but in the end I either mess up malloc and cause gcov to not spit out my coverage data or I don't have a way to have cmocka run malloc cases and return real memory i.e., not reeturn from mock() calls. On one hand I could call real malloc from my test driver but and pass that to will_return but my test_code doesn't know the size of the memory needed, only the code under test knows that.

given time constraints I don't want to move away from cmocka and my current test infrastructure. I'd consider other ideas in the future though if what I want isn't possible. What I'm looking for I know isn't new but I'm trying to use a cmocka/gcov solution.

Thanks

score 2 · Answer 1 · answered Dec 30 '15 at 05:44

This all comes down to what symbols I was messing with, either using -lW,--wrap or clever #defines. In either case I was either clobbering the symbol for other call sites and breaking code or confusing cmocka with not dequeuing queued up returns.

Also the reason my gcda files were not being generated correctly is my attempts to use -Wl,--wrap=fseek and cmocka's mock() was messing me up.

A clever #define on fseek/malloc/etc combined with mock() for a symbol that gets called in your wrapper implementation can in short query the test suite to see if you should return something bogus to cause the test to fail or return the real results. A bit hacky but does the trick.

I should add there are probably better ways to do this and I'm interested in better solutions if anyone wants to share. — railed mail, Dec 30 '15 at 05:44
Please, don't be afraid to award yourself the answer. It will make things clearer for others who find the queston in future. — Mawg says reinstate Monica, Jun 09 '16 at 08:57

score 0 · Answer 2 · answered Aug 31 '16 at 07:45

This workaround works for me: wrap _test_malloc() instead of malloc().

Working example can be found at https://github.com/CESNET/Nemea-Framework/blob/2ef806a0297eddc920dc7ae71731dfb2c0e49a5b/libtrap. tests/test_trap_buffer.c contains an implementation of a wrap function __wrap__test_malloc() (note the 4x '_' in the name)

 void *__real__test_malloc(const size_t size, const char* file, const int line);

 void *__wrap__test_malloc(size_t size)
 {
    int fail = (int) mock();
    if (fail) {
       return NULL;
    } else {
       return __real__test_malloc(size, __FILE__, __LINE__);
    }
 }

and e.g. test_create_destroy() to test the tb_init() function which uses 3x malloc():

 static void test_create_destroy(void **state)
 {
    trap_buffer_t *b = NULL;
    (void) state; /* unused */

    b = tb_init(0, 0);
    assert_null(b);
    b = tb_init(0, 1);
    assert_null(b);
    b = tb_init(1, 0);
    assert_null(b);

    will_return(__wrap__test_malloc, 0);
    will_return(__wrap__test_malloc, 0);
    will_return(__wrap__test_malloc, 0);
    b = tb_init(10, 100000);
    assert_non_null(b);
    tb_destroy(&b);
    tb_destroy(&b);
    tb_destroy(NULL);
 }

For the completeness, tb_init() is in src/trap_buffer.c line 146.

Compilation can be run like this (sample from Makefile):

buffer:
     gcc --coverage -g -O0 -DUNIT_TESTING -c tests/test_trap_buffer.c
     gcc --coverage -g -O0 -DUNIT_TESTING -c src/trap_buffer.c
     gcc -g -O0 -Wl,--wrap=_test_malloc -lcmocka --coverage -DUNIT_TESTING -o test_buffer test_trap_buffer.o trap_buffer.o

See the UNIT_TESTING preprocessor macro defined for cmocka, this is important since it enables testing allocation functions in our code.

Finally, running the test generates *.gcda files for us, so we can visualize the code coverage. Output for the tested tb_init(): https://codecov.io/gh/CESNET/Nemea-Framework/src/775cfd34c9e74574741bc6a0a2b509ae6474dbdb/libtrap/src/trap_buffer.c#L146

cmocka malloc testing OOM and gcov

2 Answers2