2

I'm currently trying to evaluate different testing frameworks. When using mocking frameworks (I'm leaning towards FakeIt, but google mock is good too), I know that you can roll your own "performance" testing by using the OS's timer calls before and after calling a function to verify the function's performance. This is not what I'm after.

What I do have are classes that implement delays on outputs given certain inputs. For example:

  • input 1 goes from low to high
  • output 1 goes from low to high after 1.5 seconds.

I'd like to be able to do something where I specify a boundary: 

myMock.theInput();
EXPECT_CALL(myMock, theDelayedOutput())
  .Times(1)
  .Before(1.6sec)
  .After(1.4sec);

For clarification, the Before and After lines are not supported. This is just an example of what I'd prefer as an easy syntax.

Is it possible to just implement a "delay" function within windows between making the input call and before checking EXPECT_CALL?

That's part of the way - I'd still have to then start a proprietary timer. Something like this?

myMock.theInput();
windowSleep(1.4);
startTimer();
EXPECT_CALL(myMock, theDelayedOutput())
  .Times(1)
endTimer();
ASSERT_TRUE(elapsedTime() <= 0.2);
273K
  • 29,503
  • 10
  • 41
  • 64
Jeff Lamb
  • 5,755
  • 4
  • 37
  • 54

2 Answers2

2

Since you tagged the question with unit-testing: In unit-testing, your tests should never depend on the passing of physical time. This has various reasons: One being that you want your tests to run as quickly as possible, so you don't want delays. Another reason being that the timing on the development environment (where your unit-tests run) can be completely different from the target environment, because of different hardware, operating systems, system load, ...

That is, tests involving the elapsing of physical time do make sense, but they would be integration tests on the target system (or some simulation thereof). In unit-testing you would take a different approach:

You would mock the functions/methods that represent your clock. For example, if your function funcA sets a timer via setTimer, providing a callback to be called when the time has elapsed:

  • In some of your tests you mock the setTimer function, call funcA and, as part of the tests' pass criterion, check that the mocked setTimer was called with the arguments you expected.
  • In some other tests, you call the callback function directly from the test to see if that function behaves correctly.

In both of these unit-test cases, you don't wait for the physical time to elapse.

That said, you certainly can use the same testing frameworks that are used for unit-testing also for integration testing (even if their names, like JUnit, seem to indicate that they are for unit-testing only). And, if integration testing is actually the kind of testing you are aiming at, suggestions as from Jeff Lamb will definitely be of help.

Dirk Herrmann
  • 5,550
  • 1
  • 21
  • 47
1

I'm pretty sure this solution will work for any framework without modifying the framework:

myMock.theInput();
startTimer();
EXPECT_CALL(myMock, theDelayedOutput());
endTimer();
ASSERT_TRUE(elapsedTime() >= 1.4);
ASSERT_TRUE(elapsedTime() <= 1.6);

This can then be wrapped in a macro:

#define EXPECT_CALL_DELAYED(theMock, expectCall, lowerBound, upperBound) {\
  startTimer();\
  EXPECT_CALL(theMock, expectCall);\
  endTimer();\
  ASSERT_TRUE(elapsedTime() >= lowerBound);\
  ASSERT_TRUE(elapsedTime() <= upperBound);\
}

The final test code is then:

myMock.theInput();
EXPECT_CALL_DELAYED(myMock, theDelayedOutput(), 1.4, 1.6);

Alternately, you can adhere to the DRY principle and pre-specify a "window" of timing. This allows you to test, specifying exact timing, but without the downside of repeating yourself having to add a 0.1 second buffer up and down every time.

EXPECT_CALL_DELAYED_SET_WINDOW(0.1);
myMock.theInput();
EXPECT_CALL_DELAYED(myMock, theDelayedOutput(), 1.5);
myMock.theSecondInput();
EXPECT_CALL_DELAYED(myMock, theSecondDelayedOutput(), 3.1);

I haven't tested any of this, though. I'll update later and accept if this works.

Jeff Lamb
  • 5,755
  • 4
  • 37
  • 54