Unit testing c++. How to test private members?

Question

I would like to make unit tests for my C++ application.

What is the correct form to test private members of a class? Make a friend class which will test the private members, use a derived class, or some other trick?

Which technique does the testing APIs use?

Answer 1

Typically, one only tests the public interface as discussed in the question's comments.

There are times however when it is helpful to test private or protected methods. For example, the implementation may have some non-trivial complexities that are hidden from users and that can be tested more precisely with access to non-public members. Often it's better to figure out a way to remove that complexity or figure out how to expose the relevant portions publicly, but not always.

One way to allow unit tests access to non-public members is via the friend construct.

Answer 2

Answering this question touches many other topics. Beside any religiosity in CleanCode, TDD and others:

There are several ways to access private members. In any case you have to overrule the tested code! This is possible on both levels of parsing C++ (preprocessor and language itself):

Define all to public

By using the preprocessor you are able to break encapsulation.

#define private public
#define protected public
#define class struct

The disadvantage is, that the class of the delivered code is not the same as in the test! The C++ Standard in chapter 9.2.13 says:

The order of allocation of non-static data members with different access control is unspecified.

This means, that the compiler has the right to reorder the member variables and virtual functions for the test. You may struggle, that this won't harm your classes if no buffer overflow happens, but it means, that you won't test the same code as you deliver. It means, that if you access members of an object, that was initialized by code, compiled with private not defined to public, the offset of your member may differ!

Friends

This method needs to change the tested class for befriending it with the test class or the test function. Some testing frameworks like gtest (FRIEND_TEST(..);) have special functionality to support this way of accessing private things.

class X
{
private:
    friend class Test_X;
};

It opens the class only for the test and does not open up the world, but you have to modify the code that gets delivered. In my opinion this is a bad thing, because a test should never change the tested code. As a further disadvantage it gives other classes of the delivered code the possibility to intrude your class by naming themselves like a test class (this would also harm the ODR rule of the C++ Standard).

Declaring the private things protected and derive from the class for tests

Not a very elegant way, very intrusive, but works also:

class X
{
protected:
    int myPrivate;
};

class Test_X: public X
{
    // Now you can access the myPrivate member.
};

Any other way with macros

Works, but has the same disadvantages on standard conformity like the first way. e.g.:

class X
{
#ifndef UNITTEST
private:
#endif
};

I think that the last both ways are no alternatives to the first two ways, because they have no advantages over the first ones, but are more intrusive on the tested code. The first way is very risky, so you may use the befriending approach.

---

Some words on the never-test-private-things-discussion. One of the upsides of unit testing at all is, that you will reach very early the point, where you have to improve the design of your code. This is also sometimes one of the downsides of unit testing. It makes object orientation sometimes more complicated, than it has to be. Especially if you follow the rule to design classes in the same way the real world objects are.

Then you have to change the code sometimes into something ugly, because the unit testing approach forces you to do so. Working on complex frameworks, that are used to control physical processes, is one example. There you want to map the code on the physical process, because often parts of the process are already very complex. The dependency list on that processes gets sometimes very long. This is one possible moment, where testing private members is getting nice. You have to trade-off with the advantages and disadvantages of each approach.

Classes are getting sometimes complex! Then you have to decide to split them or to take them as they are. Sometimes the second decision makes more sense. In the end it is always a question of which goals you want to achieve (e.g. perfect design, quick incorporation times, low development costs...).

---

My Opinion

My decision process for accessing private members looks like this:

1. Do you need to test private members themselves? (Often this reduces the total number of tests needed)
2. If yes, do you see any design advantage to refactor the class?
3. If no, befriend the test in your class (use this because of the missing alternatives).

I don't like the befriending approach, because it changes the tested code, but the risk to test something, that may not be the same as delivered (as possible with the first approach), will not justify the cleaner code.

BTW: Testing only the public interface is also a fluent matter, because in my experience it changes as often as the private implementation does. So you have no advantage to reduce the test on public members.

Answer 3

I haven't found a golden solution myself, but you can use friend to test private members, if you know how the testing framework names it's methods. I use the following to test private members with Google test. While this works quite well, note that it's a hack, and I don't use it in production code.

In the header of the code I want to test (stylesheet.h), I have:

#ifndef TEST_FRIENDS
#define TEST_FRIENDS
#endif

class Stylesheet {
TEST_FRIENDS;
public:
    // ...
private:
    // ...
};

and in the test I have:

#include <gtest/gtest.h>

#define TEST_FRIENDS \
    friend class StylesheetTest_ParseSingleClause_Test; \
    friend class StylesheetTest_ParseMultipleClauses_Test;

#include "stylesheet.h"

TEST(StylesheetTest, ParseSingleClause) {
    // can use private members of class Stylesheet here.
}

You always add a new line to TEST_FRIENDS if you add a new test that accesses private members. The benefits of this technique are that it is fairly unobstrusive in the tested code, as you only add a few #defines, which have no effect when not testing. The downside is that it is a bit verbose in the tests.

Now one word as to why you would want to do this. Ideally of course, you have small classes with well-defined responsibilities, and the classes have easily testable interfaces. However, in practice that's not always easy. If you are writing a library, what is private and public is dictated by what you want the consumer of the library to be able to use (your public API), and not by what's in need of testing or not. You can have invariants that are very unlikely to change, and need to be tested, but are of no interest to the consumer of your API. Then, black-box-testing of the API is not enough. Also if you encounter bugs and write additional tests to prevent regressions, it can be neccessary to test private stuff.

Answer 4

The desire to test private members is a design smell, generally indicating that there is a class trapped inside your class struggling to get out. All of the functionality of a class should be exercisable through its public methods; functionality that can't be accessed publicly doesn't actually exist.

There are a couple of approaches to realizing that you need to test that your private methods do what they say on the tin. Friend classes are the worst of these; they tie the test to the implementation of the class under test in a way that is prima facie fragile. Somewhat better is dependency injection: Making the private methods' dependencies class attributes that the test can supply mocked-up versions of so as to allow the testing of private methods through the public interface. Best is to extract a class that encapsulates the behavior your private methods have as its public interface, and then test the new class as you normally would.

For more details, consult Clean Code.

Answer 5

Sometimes, it is required to test private methods. Testing can be done by adding FRIEND_TEST to the class.

// Production code
// prod.h

#include "gtest/gtest_prod.h"
...   

class ProdCode 
{
    private:
    FRIEND_TEST(ProdTest, IsFooReturnZero);
    int Foo(void* x);
};

//Test.cpp
// TestCode
...
TEST(ProdTest, IsFooReturnZero) 
{
    ProdCode ProdObj;
    EXPECT_EQ(0, ProdObj.Foo(NULL)); //Testing private member function Foo()
}

Adding more info, since many are not aware of gtest features.

This is from gtest/gtest_prod.h:

// Copyright 2006, Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
//     * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
//     * Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following disclaimer
// in the documentation and/or other materials provided with the
// distribution.
//     * Neither the name of Google Inc. nor the names of its
// contributors may be used to endorse or promote products derived from
// this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

//
// Google C++ Testing and Mocking Framework definitions useful in production code.
// GOOGLETEST_CM0003 DO NOT DELETE

#ifndef GTEST_INCLUDE_GTEST_GTEST_PROD_H_
#define GTEST_INCLUDE_GTEST_GTEST_PROD_H_

// When you need to test the private or protected members of a class,
// use the FRIEND_TEST macro to declare your tests as friends of the
// class.  For example:
//
// class MyClass {
//  private:
//   void PrivateMethod();
//   FRIEND_TEST(MyClassTest, PrivateMethodWorks);
// };
//
// class MyClassTest : public testing::Test {
//   // ...
// };
//
// TEST_F(MyClassTest, PrivateMethodWorks) {
//   // Can call MyClass::PrivateMethod() here.
// }
//
// Note: The test class must be in the same namespace as the class being tested.
// For example, putting MyClassTest in an anonymous namespace will not work.

#define FRIEND_TEST(test_case_name, test_name)\
friend class test_case_name##_##test_name##_Test

#endif  // GTEST_INCLUDE_GTEST_GTEST_PROD_H_

Answer 6

Notwithstanding the comments regarding the appropriateness of testing private methods, suppose you really need to... this is often the case for example when working with legacy code prior to refactoring it into something more appropriate. Here is the pattern i've used:

// In testable.hpp:
#if defined UNIT_TESTING
#   define ACCESSIBLE_FROM_TESTS : public
#   define CONCRETE virtual
#else
#   define ACCESSIBLE_FROM_TESTS
#   define CONCRETE
#endif

Then, within the code:

#include "testable.hpp"

class MyClass {
...
private ACCESSIBLE_FROM_TESTS:
    int someTestablePrivateMethod(int param);

private:
    // Stuff we don't want the unit tests to see...
    int someNonTestablePrivateMethod();

    class Impl;
    boost::scoped_ptr<Impl> _impl;
}

Is it better than defining test friends? It seems less verbose than the alternative, and is clear within the header what is happening. Both solutions have nothing to do with security: if you are really concerned about the methods or members, then these need to be hidden inside an opaque implementation possibly with other protections.

Answer 7

There's a simple solution in C++ using #define. Just wrap the include of your "ClassUnderTest" like this:

#define protected public
#define private   public

#include <ClassUnderTest.hpp>

#undef protected
#undef private

[Credit goes to this article and RonFox][1]

Answer 8

Another solution that doesn't require you to add any gtest headers or macros to your production code could be to mark the private members as protected and create a derived class used in your tests with public functions exposing the private members.

Class to be tested:

#include <cstdint>

class Widget {
protected:
    uint8_t foo;
    uint8_t bar;
    void baz();
}

And the test class:

#include "gtest/gtest.h"
#include <cstdint>
#include "../src/widget.h"

// create a "test" class derived from your production class

class TestWidget : public Widget {
public:
    // create public methods that expose the private members
    uint8_t getFoo() {
        return foo;
    }

    void callBar() {
        bar();
    }
}

TEST(WidgetTests, TestFoo) {
    // Use the derived class in your tests
    TestWidget widget;

    // call a protected method
    widget.callBar();

    // check the value of a protected member
    EXPECT_EQ(widget.getFoo(), 10);
}

Of course the main caveat here is that you must mark the private members you need to access as protected, but it's another option if you want to keep the "test" code out of your production classes.

Answer 9

I think a better solution is to implement a code generator. When supplied with a class name or a directory path with classes to test, it should scan the source, identify the private data, and generate a new temporary directory with the generated source. The test should then use that class or that path for executing the tests.

It must preserve the member layout if possible; without that the tests would be redundant.

The advantage here is that there is no intrusion on the source itself, and the generated directory can be safely ignored by version control. No reflection necessary.

The disadvantage is that the size of this directory is the same or more than the input source, so large projects which need high coverage may find it cumbersome.

So what do we do in these new files?

1. Insert the test class as a friend. Tests will have access to the class internals wholesale. May mess with data layout(?).
2. Make the privates public. Same as above but may mess with the member data layout.
3. Insert some friend or member accessors for the class (I am not sure if mutators should be made because that might not be useful unless stress testing perhaps). Tests only have access to what's visible, and in a language like C++ which doesn't access-qualify members individually, this allows for granular exposition of data, without necessarily messing with the data layout(??).

Taken to an end, we will get something like Unreal Engine's Header tool that parses custom macro-like metadata into boilerplate or generated code which is added to the project.

Answer 10

I would prefer adding -Dprivate=public option in Makefile of unit-test, avoiding to modify anything in my original projects