GoogleTest: How to skip a test?

Question

Using Google Test 1.6 (Windows 7, Visual Studio C++). How can I turn off a given test? (aka how can I prevent a test from running). Is there anything I can do besides commenting out the whole test?

Answer 1

The docs for Google Test 1.7 suggest:

If you have a broken test that you cannot fix right away, you can add the DISABLED_ prefix to its name. This will exclude it from execution. This is better than commenting out the code or using #if 0, as disabled tests are still compiled (and thus won't rot).

Example from the above documentation:

// Tests that Foo does Abc.
TEST(FooTest, DISABLED_DoesAbc) { ... }

class DISABLED_BarTest : public testing::Test { ... };

// Tests that Bar does Xyz.
TEST_F(DISABLED_BarTest, DoesXyz) { ... }

If you have access to more recent versions of Google Test (the current version is v1.12.1), check out the GTEST_SKIP() macro suggested by jslmsca in the comments and Peter Bloomfield in another answer. From the sample in advanced.md:

TEST(SkipTest, DoesSkip) {
  GTEST_SKIP() << "Skipping single test";
  EXPECT_EQ(0, 1);  // Won't fail; it won't be executed
}

class SkipFixture : public ::testing::Test {
 protected:
  void SetUp() override {
    GTEST_SKIP() << "Skipping all tests for this fixture";
  }
};

// Tests for SkipFixture won't be executed.
TEST_F(SkipFixture, SkipsOneTest) {
  EXPECT_EQ(5, 7);  // Won't fail
}

Answer 2

You can also run a subset of tests, according to the documentation:

Running a Subset of the Tests

By default, a Google Test program runs all tests the user has defined. Sometimes, you want to run only a subset of the tests (e.g. for debugging or quickly verifying a change). If you set the GTEST_FILTER environment variable or the --gtest_filter flag to a filter string, Google Test will only run the tests whose full names (in the form of TestCaseName.TestName) match the filter.

The format of a filter is a ':'-separated list of wildcard patterns (called the positive patterns) optionally followed by a '-' and another ':'-separated pattern list (called the negative patterns). A test matches the filter if and only if it matches any of the positive patterns but does not match any of the negative patterns.

A pattern may contain '*' (matches any string) or '?' (matches any single character). For convenience, the filter '*-NegativePatterns' can be also written as '-NegativePatterns'.

For example:

./foo_test Has no flag, and thus runs all its tests.
./foo_test --gtest_filter=* Also runs everything, due to the single match-everything * value.
./foo_test --gtest_filter=FooTest.* Runs everything in test case FooTest.
./foo_test --gtest_filter=*Null*:*Constructor* Runs any test whose full name contains either "Null" or "Constructor".
./foo_test --gtest_filter=-*DeathTest.* Runs all non-death tests.
./foo_test --gtest_filter=FooTest.*-FooTest.Bar Runs everything in test case FooTest except FooTest.Bar. 

Not the prettiest solution, but it works.

Answer 3

You can now use the GTEST_SKIP() macro to conditionally skip a test at runtime. For example:

TEST(Foo, Bar)
{
    if (blah)
        GTEST_SKIP();

    ...
}

Note that this is a very recent feature so you may need to update your GoogleTest library to use it.

Answer 4

Here's the expression to include tests whose names have the strings foo1 or foo2 in them and exclude tests whose names have the strings bar1 or bar2 in them:

--gtest_filter=*foo1*:*foo2*-*bar1*:*bar2*

Answer 5

I prefer to do it in code:

// Run a specific test only
//testing::GTEST_FLAG(filter) = "MyLibrary.TestReading"; // I'm testing a new feature, run something quickly

// Exclude a specific test
testing::GTEST_FLAG(filter) = "-MyLibrary.TestWriting"; // The writing test is broken, so skip it

I can either comment out both lines to run all tests, uncomment out the first line to test a single feature that I'm investigating/working on, or uncomment the second line if a test is broken but I want to test everything else.
You can also test/exclude a suite of features by using wildcards and writing a list, "MyLibrary.TestNetwork*" or "-MyLibrary.TestFileSystem*".

Answer 6

If more than one test are needed be skipped

--gtest_filter=-TestName.*:TestName.*TestCase

Answer 7

For another approach, you can wrap your tests in a function and use normal conditional checks at runtime to only execute them if you want.

#include <gtest/gtest.h>

const bool skip_some_test = true;

bool some_test_was_run = false;

void someTest() {
   EXPECT_TRUE(!skip_some_test);
   some_test_was_run = true;
}

TEST(BasicTest, Sanity) {
   EXPECT_EQ(1, 1);
   if(!skip_some_test) {
      someTest();
      EXPECT_TRUE(some_test_was_run);
   }
}

This is useful for me as I'm trying to run some tests only when a system supports dual stack IPv6.

Technically that dualstack stuff shouldn't really be a unit test as it depends on the system. But I can't really make any integration tests until I have tested they work anyway and this ensures that it won't report failures when it's not the codes fault.

As for the test of it I have stub objects that simulate a system's support for dualstack (or lack of) by constructing fake sockets.

The only downside is that the test output and the number of tests will change which could cause issues with something that monitors the number of successful tests.

You can also use ASSERT_* rather than EQUAL_*. Assert will about the rest of the test if it fails. Prevents a lot of redundant stuff being dumped to the console.

Answer 8

I had the same need for conditional tests, and I figured out a good workaround. I defined a macro TEST_C that works like a TEST_F macro, but it has a third parameter, which is a boolean expression, evaluated runtime in main.cpp BEFORE the tests are started. Tests that evaluate false are not executed. The macro is ugly, but it look like:

#pragma once
extern std::map<std::string, std::function<bool()> >* m_conditionalTests;
#define TEST_C(test_fixture, test_name, test_condition)\
class test_fixture##_##test_name##_ConditionClass\
{\
    public:\
    test_fixture##_##test_name##_ConditionClass()\
    {\
        std::string name = std::string(#test_fixture) + "." + std::string(#test_name);\
        if (m_conditionalTests==NULL) {\
            m_conditionalTests = new std::map<std::string, std::function<bool()> >();\
        }\
        m_conditionalTests->insert(std::make_pair(name, []()\
        {\
            DeviceInfo device = Connection::Instance()->GetDeviceInfo();\
            return test_condition;\
        }));\
    }\
} test_fixture##_##test_name##_ConditionInstance;\
TEST_F(test_fixture, test_name)

Additionally, in your main.cpp, you need this loop to exclude the tests that evaluate false:

// identify tests that cannot run on this device
std::string excludeTests;
for (const auto& exclusion : *m_conditionalTests)
{
    bool run = exclusion.second();
    if (!run)
    {
        excludeTests += ":" + exclusion.first;
    }
}

// add the exclusion list to gtest
std::string str = ::testing::GTEST_FLAG(filter);
::testing::GTEST_FLAG(filter) = str + ":-" + excludeTests;

// run all tests
int result = RUN_ALL_TESTS();