4

I would like to write unit tests for my embedded application software using Google Tests.

These tests would be performed on the application software, which is written in C++. The drivers being used by the application software (eg. I2C, SPI), fault assertion are written in C. My questions would be:

  1. What would be a good place to start off? I mean resources I could read to learn more about using Google Test in an embedded environment.
  2. How do I go about mocking my driver files? For example, if I have a void read(uint8_t address) function, within my I2C library, how do I go about mocking this function, so that this particular function is called within my C++ class?
  3. These driver files written in C are also included in my C++ files. I tried compiling a bare Test file, only including my C++ class header, and had compilation issues, since the compiler couldn't find the driver header. How can I avoid this issue?
  4. Managing failed assertions with the code - Failed assertions within my driver library, calls for a system reset. How can I emulate this within the tests?
mmcblk1
  • 158
  • 1
  • 3
  • 10
  • How would you go about sending output from GoogleTest in your embedded system to your PC? – Thomas Matthews Mar 05 '20 at 21:51
  • 1
    One idea is to replace the call that reads I2C with a function that reads from a file. This would be a *stub* or mocking the I2C function. – Thomas Matthews Mar 05 '20 at 21:53
  • 2
    I highly recommend performing the tests and mocking on the PC. – Thomas Matthews Mar 05 '20 at 21:53
  • 1
    I am going along those line. The tests would be done on the PC, rather than the microcontroller. I was going thorough this resource - [Using GoogleTest and GoogleMock frameworks for embedded C](https://www.codeproject.com/Articles/1040972/Using-GoogleTest-and-GoogleMock-frameworks-for-emb). Whereas here the application was written in C, mine happens to be in C++, so its a bit confusing figuring out the right approach in my case. – mmcblk1 Mar 06 '20 at 06:29
  • Is this actually an embedded system? It sounds like some PC in disguise, "embedded Linux" or something like that. Or...? – Lundin Mar 06 '20 at 10:38
  • The code runs on an M4-Cortex processor. – mmcblk1 Mar 06 '20 at 11:06

2 Answers2

10

I've recently tested a FAT file system and bootloader implementation with gTest (GoogleTest) for an Arm Cortex-M3 core, so I'll leave my two cents.

Embedded software testing presents the problem that it's impossible to replicate the HW environment through mocking. I came up with three sets of tests:

A) Unit tests (that I use in TDD) that run on my PC. I use these tests to develop my application logic. This is where I need mocking/stubbing. My company uses a hardware abstraction layer (HAL), and that's what I mock. This last bit is fundamental if you want to write testable code.

/* this is not testable */
my_register->bit0 = 1;

/* this is also not testable */
*my_register |= BIT0;

Don't do direct register access, use a simple HAL wrapper function that can be mocked:

/* this is testable */
void set_bit(uint32_t* reg, uint8_t bit)
{
    *reg |= bit;
}

set_bit(my_register , BIT0);

The latter is testable because you're going to mock the set_bit function, thus breaking the dependency on the HW.

B) Unit tests on the target. This is a much smaller set of tests than (A), but it's still useful especially for testing drivers and the HAL functions. The idea behind these tests is that I can properly test the functions that I'll mock. Because this runs on the target, I need this as simple and lightweight as possible, so I use MinUnit, which is a single C header file. I've run on-target tests with MinUnit on a Cortex-M3 core and on a proprietary DSP code (without any modifications). I've also used TDD here.

C) Integration tests. I use Python and Behave here, to build, download, and run the whole application on the target.

Answering your questions:

  1. As others have already said, start with gTest Primer, and don't worry about mocking, just getting the hang of using gTest. A good alternative that offers some memory checking (for leaks) is Cpputest. I have a slight preference for the gTest syntax for deriving the setup classes. Cpputest can run tests written with gTest. Both are great frameworks.

  2. I used Fake Function Frakework for mocking and stubbing. It's ridiculously simple to use and it offers everything expected from a good mocking framework: setting different return values, passing callbacks, checking the argument call history, etc. I want to give Ceedling a go. So far FFF has been great.

  3. I don't do that. I compile the testing framework and my tests with a C++ compiler (g++ in my case) and my embedded code with a C compiler (gcc), and just link them together. From the example below, you'll see that I don't include C++ headers in C files. When linking your tests, you'll link everything but the C source files for the functions you're mocking.

Managing failed assertions with the code - Failed assertions within my driver library, calls for a system reset. How can I emulate this within the tests?

I'd mock the reset function, adding a callback to "reset" whatever you need.

Say that you want to test a read_temperature function that uses the read function. Below is a gTest example that uses FFF for mocking.

hal_i2c.h

/* HAL Low-level driver function */

/**
 * @brief Read a byte from the I2C bus.
 * 
 * @param address The I2C slave address.
 * @return uint8_t The data read from the I2C bus.
 */
uint8_t read(uint8_t address);

read_temperature.h

/**
 * @brief Read the temperature from the sensor in the board
 * 
 * @return float The temperature in degrees Celsius.
 */
float read_temperature(void);

read_temp.c

#include <hal_i2c.h>

float read_temperature(void)
{
    unit8_t raw_value;
    float temp;

    /* Read the raw value from the I2C sensor at address 0xAB */
    raw_value = read(0xAB);

    /* Convert the raw value to Celcius */
    temp = ((float)raw_value)/0.17+273;
    return temp;
}

test_i2c.cpp

#include <gtest/gtest.h>
#include <fff.h>

DEFINE_FFF_GLOBALS;

extern "C"
{
#include <hal_i2c.h>
#include <read_temperature.h>

// Declare the fake C functions using FFF. This needs be inside the extern "C"
// block because we're mocking a C function and the C++ name-mangling would
// break linking otherwise.


// Create a mock for the uint8_t read(uint8_t address) function.
FAKE_VALUE_FUNC(uint8_t , read, uint8_t);
}

TEST(I2CTest, test_read) {

    // This clears the FFF counters for the fake read() function
    RESET_FAKE(read);

    // Set the raw temperature value the fake for read should return
    read_fake.return_val = 0xAB;

    // Make sure that we read 123.4 degrees
    ASSERT_EQ((float)123.4, read_temperature());
}

For more complex test scenarios with test classes, you can call RESET_FAKE in the SetUp() method.

Hope this helps! Cheers!

Leonardo
  • 1,533
  • 17
  • 28
  • Thanks for your answer. Back then when I had asked the question, I had limited idea about stubbing/mocking and `GTest` was initially painful. So, I decided to use `CppUTest`. Now after reading your answer, I guess, I can give `GTest` another try. One question regarding the `Fake Function Frakework`. Is this compatible with `GTest`? Also, how do you emulate(mock) say when you have a lot of registers? Say, write to a register at `0xAB` and `0xC1` addresses. I ended up using a large array and writing at these indicies and later verified if the values match. – mmcblk1 Dec 11 '20 at 08:15
  • 1
    Yes, that's what I've been using: gTest+FFF. My example is exactly that. – Leonardo Dec 11 '20 at 19:04
  • In point 3 you say: `I compile the testing framework and my tests with a C++ compiler (g++ in my case) and my embedded code with a C compiler (gcc), and just link them together.` Could you pint me to a resource which explains how you do the linking them together bit? Im compiling my embedded program with gcc and my google test project with g++ and CMake. But i cant really get the test project to work without having to include the .C source files together with the header files. – Jelle Bleeker Mar 03 '23 at 01:32
  • If I remember correctly, the only difference is that instead of using `gcc` to link, I used `g++` (yes, I know that under the hood they'll call `ld`). I'd suggest that you use CMake to build the code under test and the test themselves, so it'll take care of everything for you. – Leonardo Mar 03 '23 at 16:26
1
  1. I don't know of any particular resources for using Gtest for baremetal target tests, but a good place to start in general would be to read the Gtest Primer and depending your software architecture maybe even the Gmock documentation. The latter might get useful when testing application relevent classes and not just low-level drivers.
  2. There are several options for this. The most common one I've seen so far is having two different implementations for the target and the platform running the tests. So e.g. you might have two files

    • ic2.c
    • i2c_x86.cpp

      And depending on whether you're currently compiling for the target or the test platform you use either one of them.

    Another option would be to lift the C implementation to C++ and write a class wrapper around your driver. This would allow you to benefit from C++ features and use things like dependency injection, inheritance, CRTP and so on...

  3. Not sure I understand what you're asking for.

  4. Gtest has an ASSERT_DEATH test, e.g. my current code base contains the following test

      // 2 byte message does not fit 14bit, assertion triggered
      ASSERT_DEATH(encode_datagram(make_datagram(0, 64, 0)), ".*");
Vinci
  • 1,382
  • 10
  • 12
  • 1
    Drivers are by definition hardware-specific, so you simply _can't_ test I2C on a bloody x86. _Drivers must be tested on the specific hardware_, period. You can test _hardware abstraction layers_ but that's by definition not a driver, but a wrapper around a driver. – Lundin Mar 06 '20 at 15:22
  • 1
    I am trying to test the application software and not the drivers. Hence, I was asking about mocking the drivers. I want to check if the application logic is working as desired, rather than the drivers. – mmcblk1 Mar 07 '20 at 01:26
  • 1
    @Lundin There ain't a single sentence on the whole page indicating that the he want's to test I2C drivers. Nobody is assuming that but you. He even specifically asks about mocking... – Vinci Mar 07 '20 at 06:16
  • The question isn't exactly clear, but regardless you need to test these things on the live hardware. Timing will be completely different, machine code will be completely different and so on. Testing on a PC will reveal if the PC works or not, and that's about all good it does. – Lundin Mar 09 '20 at 07:43
  • 2
    The question is perfectly clear on this regards. It states that unit tests are to be performed on **application software** written in C++. The followup sentence then says that drivers used by the application software are written in C and then one of the bullet points is specifically about how to **mock** those. – Vinci Mar 09 '20 at 11:33
  • And also, the [reference article](https://www.codeproject.com/Articles/1040972/Using-GoogleTest-and-GoogleMock-frameworks-for-emb), I had mentioned earlier, speak about this. – mmcblk1 Mar 09 '20 at 14:30
  • @Lundin, while drivers can not achieve 100 % coverage without the HW, mocking is super useful eve for driver development as all the logic can be developed in a PC platform. It's tricky but possible to test ISRs and code interaction on a PC. Been there, done that. It's just a matter of using a HAL and writing testable code. – Leonardo Dec 11 '20 at 19:06