Suppress output from google test assertions

Question

I have a couple of tests that test interpolations of continuous functions, and when my implementation fails for some reason I get lots of output like the following from my Google test suite:

/home/tlycken/exjobb/Code/alpha-orbit-follower/test/interpolation/interpolation-tests.cpp:71: Failure
The difference between this->test_function_y_deriv(x,y) and this->getInterpObjectPtr()->evaluateAt(x,y,0,1) is 1.5395837072062037, which exceeds tol, where
this->test_function_y_deriv(x,y) evaluates to -1.5395837072062037,
this->getInterpObjectPtr()->evaluateAt(x,y,0,1) evaluates to 0, and
tol evaluates to 0.01.

Since I basically loop over the interpolated interval and test with ASSERT_NEAR() in a lot of places, I'm not surprised, but it would be nice to suppress the details of each error message, and just see the pass/fail status of the test case in the report. If a test fails, and I don't understand why, I can reenable the output and look closer.

However, I can't find any information on how to suppress that kind of output. Perhaps my Google-fu is inadequate...

Is there a command line flag, or some other means, to suppress the output from individual ASSERT* calls, and just show the final test report?

Code sample:

As this seems to be difficult to reproduce, this is the code I'm running. In my test fixture, I have the following method:

void on_interpolated_grid(std::function<void(double)> f) {
    double dxp = dx / 10;
    double xmax = xmin + (N - 1) * dx;

    for (double xp = xmin + 2 * dx; xp < xmax - 2 * dx; xp += dxp) {
        f(xp);
    }
}

and the test is defined like this:

TYPED_TEST(Interp1D_F, SecondOrderDerivative) {
    Interp1D itp(this->N, this->x, this->f);

    this->on_interpolated_grid([itp](double x) -> void {
        ASSERT_NEAR(-sin(x), itp.evaluateAt(x, 2), 1e-3);
    });
}

The members N, x and f on the test fixture are just parameters for initialization of the interpolation object.

I also tried to wrap the call to on_interpolated_grid in ASSERT_NO_FATAL_FAILURES, but it didn't help.

@nouney: No; that only lets me add to the output, or to change how my own types are presented, but not *disable* certain types of output. Try it yourself: See if you can make a test containing `ASSERT_NEAR(3.5, 3.0, 0.1);` fail without outputting the details on the fact that the difference between 3.5 and 3.0 is greater than 0.1 - I just want to see the test fail in the summary. — Tomas Aschan, Mar 31 '14 at 10:51
Do you really use ASSERT_NEAR or EXPECT_NEAR? ASSERT_NEAR should fail the test at the first failure? — Philipp, Mar 31 '14 at 10:54
isn't it easier to filter the output to only display the first line for each result (or whatever you prefer) ? That would also give you the added advantage not to have to re-run the tests to see the details (just disable the filter). — Sander De Dycker, Apr 02 '14 at 14:34
@TomasLycken : depending on how fancy you want to get, a simple [grep](http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html) (or similar) should do the trick. — Sander De Dycker, Apr 02 '14 at 16:04
@TomasLycken: with TEST(MyTest, ManyAsserts) { for (int i=0;i<100;i++) { ASSERT_NEAR(3.0,3.5,0.1); } } I only get one message and not 100. Maybe I didn't understand your problem correctly - do you have many different tests failing or one test failing with many messages? — Philipp, Apr 03 '14 at 06:53
@SanderDeDycker: If I'd use grep to filter the test results, I'd loose all the color highlights from GTest, right? At least I don't know any way to preserve it. And that would really be a reduction in readability of the summary. — Tomas Aschan, Apr 03 '14 at 07:29
@Philipp: Please see the updated question for an excerpt of my code. Could the fact that each assertion is inside its own method call be the problem? — Tomas Aschan, Apr 03 '14 at 07:30
@TomasLycken: Thanks, that definitely clarified the question! Please see my reply and let me know if that's an appropriate solution! — Philipp, Apr 04 '14 at 12:45
I just received a downvote on this question, but I can't really see why. If I did something bad, maybe a comment to let me know would be a nice idea? — Tomas Aschan, Apr 09 '14 at 07:38

Philipp · Accepted Answer · 2014-04-07T07:39:58.727

As far as I can see, the problem results from the fact that ASSERT_NEAR behaves more like EXPECT_NEAR in this case.

The reason for this behavior is that Google Test uses return but no exceptions in ASSERT_*. See their FAQ for why they do this (in short: it allows to run tests even with exceptions disabled but has other advantages in some cases like ASSERT in try-blocks.

The downside of this approach is exactly what you have here: ASSERT_ does not propagate fatal failures if used in subfunctions (or a lambda as in your case). The GTest Advanced topics page has a section regarding this problem. With HasFatalFailure() your situation can be solved with a modified lambda method:

#include "gtest/gtest.h"
#include "gtest/gtest-typed-test.h"
#include <functional>

void on_interpolated_grid(std::function<void(double)> f) {
    ///ADDED a few values here
    double dxp = 1.0 / 10.0;
    double xmin = 0.1;
    double dx = 0.1;
    int N = 100;
    double xmax = xmin + (N - 1) * dx;

    for (double xp = xmin + 2 * dx; xp < xmax - 2 * dx; xp += dxp) {
        f(xp);
    }
}



TEST(Interp1D_F, SecondOrderDerivative) {
    ///REMOVED. Not required for sample
    //Interp1D itp(this->N, this->x, this->f);

    on_interpolated_grid([](double x) -> void {
        ///ADDED:
        //Only check for further failures if no previous fatal failure ocurred
        if (!this->HasFatalFailure())
            ASSERT_NEAR(-sin(x),0 , 1e-3);

    });
}

To see the full test output (if desired), you just remove the added line.

I had to change the call to `this->HasFatalFailure()` (i.e. add `this->`) in order to avoid `error: there are no arguments to ‘HasFatalFailure’ that depend on a template parameter` when compiling, but after adding that (and removing `ASSERT_NO_FATAL_FAILURES`) it works just as I want it. Thanks! — Tomas Aschan, Apr 07 '14 at 07:34
good point, I have added the `this->` to the answer. Not sure if this is due to different compilers (GCC usually needs `this->` when VS doesn't) or if it is because you are using `TYPED_TEST` which might result in a template while `TEST` does not. But with `this->` it should be safe in any case! — Philipp, Apr 07 '14 at 07:41

score 0 · Answer 2 · edited May 23 '17 at 11:49

Recently I asked similar question (Reducing output of gtest, to look similar to output of cxxtest).

If you want to fully suppress the output, you can use ::testing::EmptyTestEventListener. For anything fancy, you have to implement custom listener (see for example here).

You can even add an option in main() to change the listener depending on program parameters.

You said that you want colorful output. googletest's maintainers added functions to print text in color. I know you shouldn't, but if you declare those functions, you will get access to them.

score 0 · Answer 3 · answered Apr 04 '14 at 03:14

It seems google test AssertionResult are printed to standard output by design. No built-in command line option to disable specific parts of the that output either.

If tweaking google test itself is an option, then the easiest would probably be to add a condition around the PrintTestPartResult call in PrettyUnitTestResultPrinter::OnTestPartResult, and similarly with calls to PrintFullTestCommentIfPresent in OnTestEnd/PrintFailedTests. In that case, your own option to control verbosity could be included in ParseGoogleTestFlagsOnlyImpl and kColorEncodedHelpMessage.

Otherwise (i.e. if recompiling google test is not an option), it is possible to write a TestEventListener replacement to the default PrettyUnitResultPrinter (possibly inspired from PrettyUnitResultPrinter itself), as documented in google test samples.

Suppress output from google test assertions

3 Answers3