10

I'm using XCTest and OCMock to write unit tests for an iOS app, and I need direction on how to best design a unit test that verifies that a method results in an NSTimer being started.

Code under test:

- (void)start {
    ...
    self.timer = [NSTimer timerWithTimeInterval:1.0
                                         target:self
                                       selector:@selector(tick:)
                                       userInfo:nil
                                        repeats:YES];
    NSRunLoop *runLoop = [NSRunLoop currentRunLoop];
    [runLoop addTimer:self.timer forMode:NSDefaultRunLoopMode];
    ...
}

What I want to test is that the timer is created with the right arguments, and that the timer is scheduled to run on the run loop.

I've thought of the following options that I am not happy with:

  1. Actually wait for the timer to fire. (Reason I don't like it: terrible unit testing practice. It's more like a slow integration test.)
  2. Extract the timer starting code into a private method, expose that private method in a class extension file to the unit tests, and use a mock expectation to verify that the private method gets called. (Reason I don't like it: it verifies that the method gets called, but not that the method actually sets up the timer to run correctly. Also, exposing private methods is not a good practice.)
  3. Provide a mock NSTimer to the code under test. (Reason I don't like it: can't verify that it actually gets scheduled to run because timers are started via the run loop and not from some NSTimer start method.)
  4. Provide a mock NSRunLoop and verify that addTimer:forMode: gets called. (Reason I don't like it: I'd have to provide an interface into the run loop? That seems wacky.)

Can someone provide some unit testing coaching?

Sport
  • 8,570
  • 6
  • 46
  • 65
Richard Shin
  • 663
  • 5
  • 17
  • You shouldn't have to verify that it _actually gets scheduled_ to run. The scheduling and running bits are not your code; even if you unit tested them and they failed, what could you do about it? You just need to make sure your code does what it needs to do up to that point, interfaces correctly with the framework. Therefore, I think number three is the answer -- tell a mock of the `NSTimer` class object to expect `timerWithTimeInterval:...` – jscs Jan 07 '14 at 06:33
  • Mocking `NSRunLoop` also makes sense; Mike Ash has an article that would probably be helpful there: https://www.mikeash.com/pyblog/friday-qa-2010-01-01-nsrunloop-internals.html – jscs Jan 07 '14 at 06:36
  • Josh, thanks for the guidance! I ended up thinking a lot more about the problem, and your comment on "the scheduling and running bits are not your code" reminded me that I'd read somewhere, "Don't test Apple's implementation". It didn't make sense until now.I was trying to design my test to cover all possible scenarios, but you just can't do that when you're using someone else's code to do things, all you can do is test that the interaction occurs. – Richard Shin Jan 09 '14 at 07:01
  • Glad I could be helpful. Hope you'll consider writing up an answer with the way you resolved this. – jscs Jan 09 '14 at 19:02

2 Answers2

6

Okay! It took a while to figure out how to do this. I'll explain my thought process in entirety. Sorry for the long-windedness.

First, I had to figure out exactly what I was testing. There are two things my code does: it starts a repeating timer, and then the timer has a callback that makes my code do something else. Those are two separate behaviors, which means two different unit tests.

So how do you write a unit test to verify that your code starts a repeating timer correctly? There are three things you can test for in a unit test:

  • The return value of a method
  • A change of state or behavior of the system (preferably available through a public interface)
  • The interaction your code has with some other code that you don't control

With NSTimer and NSRunLoop, I had to test for interaction because there's no way to externally verify that the timer was configured correctly. Seriously, there's no repeats property. You have to intercept the method call that creates the timer itself.

Next, I realized that I wouldn't have to touch NSRunLoop at all if I created the timer with +scheduledTimerWithTimeInterval:target:selector:userInfo:repeats, which automatically starts the timer. That's one less interaction I have to test for.

Finally, to create an expectation that +scheduledTimerWithTimeInterval:target:selector:userInfo:repeats is called, you have to mock the NSTimer class, which thankfully OCMock can do now. Here's what the test looked like:

id mockTimer = [OCMockObject mockForClass:[NSTimer class]];
[[mockTimer expect] scheduledTimerWithTimeInterval:1.0
                                            target:[OCMArg any]
                                          selector:[OCMArg anySelector]
                                          userInfo:[OCMArg any]
                                           repeats:YES];

<your code that should create/schedule NSTimer>

[mockTimer verify];

Looking at this test, I thought, "Wait, how can you actually test that the timer is configured with the correct target and selector?" Well, I finally realized that I shouldn't actually care that it's configured with a particular target and selector, I should only care that when the timer fires, it does what I need it to do. And that's a really important point for writing good, future-proof unit tests: really try hard not to rely on the private interface or implementation details because those things change. Instead, test your code's behavior that won't change, and do it through the public interface.

That brings us to the second unit test: does the timer do what I need it to do? To test this, thankfully NSTimer has -fire, which causes the timer to perform the selector on the target. Thus, you don't even need to create a fake NSTimer, or do an extract & override to create a custom mock timer, all you have to do is let it rip:

id mockObserver = [OCMockObject observerMock];
[[NSNotificationCenter defaultCenter] addMockObserver:mockObserver
                                                 name:@"SomeNotificationName"
                                               object:nil];
[[mockObserver expect] notificationWithName:@"SomeNotificationName"
                                     object:[OCMArg any]];
[myCode startTimer];

[myCode.timer fire];

[mockObserver verify];
[[NSNotificationCenter defaultCenter] removeObserver:mockObserver];

A few comments about this test:

  • When the timer fires, the test expects that a NSNotification is posted to the default NSNotificationCenter. OCMock managed not to disappoint: notification broadcast testing is this easy.
  • To actually trigger the timer to fire, you need a reference to the timer. My class under test does not expose the NSTimer in a public interface, so the way I did it was by creating a class extension that exposed the private NSTimer property to my tests, as described in this SO post.
Community
  • 1
  • 1
Richard Shin
  • 663
  • 5
  • 17
  • I'm something of a TDD-newbie, so there may be something that I'm overlooking, but I'm not convinced your test is accomplishing anything. Sure, when you run it, you get a green mark, but basically you have verified that an NSTimer actually works as advertised. That's useful when you're a tester at Apple, but what does it tell you about your code? As far as I can see, nothing. In your app you could misspell the timer's selector, specify the wrong target, fire the timer at the wrong moment and forget to invalidate it, but according to the test above you're doing just great. – Elise van Looij Jan 28 '15 at 14:49
  • The point of the test is not to verify that the NSTimer does what it does -- but that when it does what it does, your code under test (CUT) does the right thing. In a *good* unit test, you want to verify that it does the right *public* thing -- mutate a public variable, change its state in a publicly noticeable way, invoke a method on a third-party object, return the right value, or in this case, broadcast the right notification. – Richard Shin Jan 29 '15 at 23:16
  • You could misspell the timer's selector, sure, but if you did, it wouldn't broadcast a notification. Or you could schedule it to run every 10 seconds instead of every second, but that's why I set up the mock timer to expect it to receive scheduledTimerWithTimeInterval:1.0. Ultimately, unit tests can and will not provide 100% reassurance that you've done everything correctly. I mean, even expecting scheduledTimerWithTimeInterval: is a pretty brittle and fragile unit test because someone can create a timer with another initializer. That's where integration/functional tests should fill the holes – Richard Shin Jan 29 '15 at 23:18
1

I really liked Richard's first approach, I expanded the code a little to use block invocations to avoid having to reference the private NSTimer property.

[[mockAPIClient expect] someMethodSuccess:[OCMIsNotNilConstraint constraint]
                                  failure:[OCMIsNotNilConstraint constraint];

id mockTimer = [OCMockObject mockForClass:[NSTimer class]];
[[[mockTimer expect] andDo:^(NSInvocation *invocation) {
    SEL selector = nil;
    [invocation getArgument:&selector atIndex:4];
    [testSubject performSelector:selector];
}] scheduledTimerWithTimeInterval:10.0
                           target:testSubject
                         selector:[OCMArg anySelector]
                         userInfo:nil
                          repeats:YES];

[testSubject viewWillAppear:YES];

[mockTimer verify];
[mockAPIClient verify];
psobko
  • 1,548
  • 1
  • 14
  • 24
  • Nice! Didn't realize you could grab arguments sent to the mock object that way. It's an improvement to not have to dig into a class's private implementation. I do think it comes at the cost of test readability, but like most things, it's a tradeoff. – Richard Shin Jan 29 '14 at 02:17