block callback not being called from a unit test program in IOS

Question

I have the following setup. I have a object called "View" in which I want to unit test a method which contains two dispatch_async calls with in it.

view.m

typedef void (^onTaskCompletion)();  //defining the block

-(void) viewdidLoad
{

    onTaskCompletion block = ^{
        // callback into the block };

    [self test1:block];

}

-(void) test1:(onTaskCompletion) block
{

    //do something   
    dispatch_async(queue, ^{

    // dispatch async into serial queue & do something

        dispatch_async(dispatch_get_main_queue){

         // calling the block

         block();
        };
    }; 
}

When I run the IOS APP , the block in -(void) viewdidLoad gets called. Works perfectly fine. But the problem I have is this:

in Tests : XCTestCase (.m fie)

@property (retain) View *view;

-(void) testMyCode
{

 onTaskCompletion block = ^{
        // Never gets called.
   };
   [view test1:block];    
}

When I try to Unit test this method test1(), The block never gets called.

Note: The break point within test1() method inside the dispatch_get_main_queue() never gets hit when running in test mode but does get hit when I just run the app. Any thoughts as to why it works when the app is run normally but not when running unit tests?

it is hard to test threading code, your unit test need to run the runloop — Bryan Chen, Dec 02 '13 at 23:47

score 1 · Answer 1 · answered Mar 03 '14 at 15:48

1

The problem you are facing is that the tests continue onwards even though they are not finished. The solution is to stall the runloop until the async test if finished.

You can use this dead-simple open source macro WAIT_WHILE(<expression>, <time_limit>) found here https://github.com/hfossli/AGAsyncTestHelper

- (void)testAsyncBlockCallback
{
    __block BOOL jobDone = NO;

    [Manager doSomeOperationOnDone:^(id data) {
        jobDone = YES; 
    }];

    WAIT_WHILE(!jobDone, 2.0);
}

answered Mar 03 '14 at 15:48

hfossli

22,616
10
116
130

You need memory barriers, otherwise the compiler can optimize your while loop to `while (!NO) {...}`. Also, waiting two seconds until the run loop expires in order to check the flag isn't optimal, too. – CouchDeveloper Mar 03 '14 at 20:25
2 seconds is just arbitrary. Could be set to minutes or hours, but every async test needs an upper limit in case the callback doesn't happen. Important detail when using buildsystems. It is just the upper limit. If it exceeds that limit it will throw XCTFail() – hfossli Mar 04 '14 at 06:57
The "timeout" issue is, that the run loop will not necessarily return when the statements in the block are executed. The run loop only returns, when there has been an event scheduled and executed on this run loop and mode. – CouchDeveloper Mar 04 '14 at 07:49
This leaves me with more questions :) How do *you* write unit tests for async operations? – hfossli Mar 04 '14 at 08:27
Ah. I found one of your answers here http://stackoverflow.com/a/21778069/202451 I'll look into it – hfossli Mar 04 '14 at 08:30
Promises can be used to make the exact issue above reliable and effective. However, Unit Testing is a more broader topic. For example, in order to test *whether* a method has been invoked *asynchronously* you can MOCK the object whose method should be invoked. You don't test the method itself, but the call-site - namely, whether it invokes the correct method under the given scenario. – CouchDeveloper Mar 04 '14 at 08:34
Vice versa, if you want to test the method that gets invoked eventually, you can MOCK the _asynchronous result provider_ that is, the ting that calls the method. Then, assert the observable side effects from the object whose methods are invoked. – CouchDeveloper Mar 04 '14 at 08:37
I have yet to experience the WAIT_WHILE macro to **not** work as expected. I don't really see the big difference using the promises pattern in your example vs the WAIT_WHILE-macro (except for code-style / structure). What is it specifically you think is wrong with the code and what is it specifically that makes your example safe? Is it the implementation of `-[RXPromise runLoopWait]`? And yes, mocking is indeed effective, but not always applicable. – hfossli Mar 04 '14 at 08:58
There are two issues in your approach: first, the compiler can optimize the while loop `while (!NO) {...}` which is `while(1){...}`. You need memory barriers, or use semaphores or std::atomic in order to avoid this. Even using the `volatile` modifier is not sufficient. I've explained the second issue already: the run loop method `runUntilDate:` and `runMode:beforeDate:` methods will ONLY return when there has been an event processed which has been scheduled on this run loop and with this mode, or when the timeout expires - which ever comes first. – CouchDeveloper Mar 04 '14 at 09:09
Thanks for answering, again. :) I have a little problem understanding what you mean by "methods will ONLY return when there has been an event processed which has been scheduled on this run loop and with this mode". 1. Every unit test runs on the main runloop, right? This macro is meant for unit testing with XCTest or SenTestingKit. 2. I can write a disclaimer noting WAIT_WHILE() should only be used from top level of the test (not within a block or on a different thread). 3. This example works http://pastie.org/8854020 – hfossli Mar 04 '14 at 10:03
It doesn't matter where the WAIT_WHILE loop is executed. When there are no events (aka methods) scheduled on the run loop with that mode, the mentioned run loop methods don't return _before_ the timeout expires. In order to ensure that the run loop returns and checks the flag, you need to schedule a method on the main run loop after you set the flag from within the completion block, so the run loop "wakes up", processes this event and returns, then - finally - you'll test the flag. – CouchDeveloper Mar 04 '14 at 20:53
Now, from the sources I can see that you repeatedly invoke the run loop with a fixed delay of 0.01 seconds internally. That is, your run loop will return every 0.01 seconds at least. So, apparently it "works", but now hogging the CPU, and creating heaps of autoreleased NSDate objects(use @autoreleaspool!). Still, there is this issue: even though clang is forced to use an indirection to obtain the value of the flag due to the `__block` modifier and thus might not be able to optimize to `while (1){..}` - it's still _undefined behavior_ according C11 §5.1.2.4 (data race); you need memory barriers – CouchDeveloper Mar 04 '14 at 22:26
I appreciate your extensive comments. I now have a good understanding of what you are criticizing. I will definitely add memory barriers. I will fix autoreleasing of NSDate. I don't really think we A) are hogging the CPU and B) that it is an issue since this is only meant for unit testing. 95% of continuous integration build time is spent on `pod install` and compiling the code and tests. The actual tests run pretty quick. I agree, it could have been better by not hogging the CPU. – hfossli Mar 05 '14 at 14:08
So I might fix it like you say, but your comments does not at all qualify for saying "this does not work". We have over 300 async tests running with this macro with very various cases and setups. Works like a charm. And we have separate unit tests for this macro as well. https://github.com/hfossli/AGAsyncTestHelper/blob/master/Tests/AGAsyncTestHelperXCTTest.m . I will add tests verifying that we don't wait until the time limit is reached unless necessary, but we would definitely notice it already if that weren't the case. – hfossli Mar 05 '14 at 14:10
All in all: thank you for pointing out the flaws even though we to some extent disagree on the severity of it. – hfossli Mar 05 '14 at 14:11

score 0 · Answer 2 · answered Dec 03 '13 at 02:48

If you want to be able to unit test asynchronous code, you will need to wrap the dispatch_async in a class that you can mock. This class would have for example:

- (void)executeInBackground:(void(^)())task;
- (void)executeInForeground:(void(^)())task;

Then, during your tests you can mock this class. Instead of actually calling the tasks, collect the tasks when they are called, and manually have them executed in your test (not actually calling asynchronously):

- (void)executeNextBackgroundTask;
- (void)executeNextForegroundTask;

Then you can explicitly test each order of execution.

block callback not being called from a unit test program in IOS

2 Answers2