Why does a false assertion in async test in GHUnit crash the app instead of just failing the test?

Question

This question has very few views and no answers yet. If you have a suggestion what to change about this question to get more eyeballs, I'd be happy to hear them. Cheers!

I'm using GHAsyncTestCase to test a custom NSOperation of mine. I'm setting the test case as a delegate on the operation object and I'm calling didFinishAsyncOperation on the main thread when it's done.

When an assertion fails it throws an exception, which ought to be caught by the test case to render the test as "failed". But instead of this expected behavior, my app get's aborted by Xcode as soon as the assertion fails.

*** Terminating app due to uncaught exception 'GHTestFailureException', reason: ''NO' should be TRUE. This should trigger a failed test, but crashes my app instead.'

I'm obviously doing something wrong. Who can tell me?

@interface TestServiceAPI : GHAsyncTestCase
@end

@implementation TestServiceAPI

    - (BOOL)shouldRunOnMainThread
    {
        return YES;
    }

    - (void)testAsyncOperation
    {
        [self prepare];

        MyOperation *op = [[[MyOperation alloc] init] autorelease];

        op.delegate = self; // delegate method is called on the main thread.

        [self.operationQueue addOperation:op];

        [self waitForStatus:kGHUnitWaitStatusSuccess timeout:1.0];
    }

    - (void)didFinishAsyncOperation
    {
        GHAssertTrue(NO, @"This should trigger a failed test, but crashes my app instead.");

        [self notify:kGHUnitWaitStatusSuccess forSelector:@selector(testAsyncOperation)];
    }

@end

I'm guessing the exception's going off in a context (different thread, eg) where there's no exception handler present to catch it. — Hot Licks, Oct 05 '11 at 20:01
Yes, that's what I'm guessing too. But this must be a standard issue scenario for async tests, right? What should I do to fix it? — epologee, Oct 05 '11 at 21:03
I'm also getting the same exception. But I use GHTestCase. When a test case failed, should it crash the app too? — Aruna, Feb 14 '14 at 12:57
No it shouldn't, the reason your app crashes is that the test suite's exception handlers don't catch your exception. Refactor your tests to not break out of the test methods. I would advise you to start using XCTest by now, or even Kiwi. — epologee, Feb 14 '14 at 16:04

score 12 · Accepted Answer · edited Jun 20 '20 at 09:12

I've been digging for a week to find a solution to this when I finally caught a break. It's been a bit weird having next to no views on a bounty question and no one cared to attempt an answer. I was thinking the question might be stupid, but there were no downvotes and no one cared to correct it either. Has StackOverflow become that saturated?

A solution.

The trick is to not assert anything from the callback method, but put the assertions back in the original test. The wait method is actually blocking the thread, which I didn't think of before. If your async callback receives any values, just store them in an ivar or property and then make assertions based on them in your original test method.

This takes care of the assertions not causing any crashes.

- (void)testAsyncOperation
{
    [self prepare];

    MyOperation *op = [[[MyOperation alloc] init] autorelease];

    op.delegate = self; // delegate method is called on the main thread.

    [self.operationQueue addOperation:op];

    // The `waitfForStatus:timeout` method will block this thread.
    [self waitForStatus:kGHUnitWaitStatusSuccess timeout:1.0];

    // And after the callback finishes, it continues here.
    GHAssertTrue(NO, @"This triggers a failed test without anything crashing.");
}

- (void)didFinishAsyncOperation
{
    [self notify:kGHUnitWaitStatusSuccess forSelector:@selector(testAsyncOperation)];
}

Awesome! Thanks for this (even though it was asked awhile ago) — MTurner, Jul 10 '12 at 03:02
Glad it was of help! One's gotta love stack overflow for archiving like this :) — epologee, Jul 10 '12 at 08:32
You are savior my friend, how I couldn't think of it, was struggling two days with that, but I think GHUnit should handle it more appropriately. I couldn't assert a boolean value inside a block code. This fixed my logic. Though I don't like so much using __block variables, it works. — George Taskos, Jul 01 '14 at 16:53
Best comment ever :D. Might I ask, why are you 'still' using GHUnit, instead of XCTest? — epologee, Jul 02 '14 at 08:01

score 2 · Answer 2 · answered Mar 29 '13 at 03:19

2

Look up your Xcode Breakpoints navigator,delete all exception breakpoints,That's all !!!

answered Mar 29 '13 at 03:19

cowry chen

11
1

Thanks for that, always forget about it. – Rostyslav Druzhchenko Feb 07 '14 at 17:37

score 0 · Answer 3 · answered Feb 19 '14 at 17:14

0

I think this question should be rather, "how to test methods with blocks in GHUnit"?

And the answer can be found here: http://samwize.com/2012/11/25/create-async-test-with-ghunit/

answered Feb 19 '14 at 17:14

Denis

1,179
10
13

score 0 · Answer 4 · answered Oct 09 '11 at 13:47

0

Looking at the header files for GHUnit, it looks like that may be what is supposed to happen with your code. A subclass of GHUnit could override this method:

// Override any exceptions; By default exceptions are raised, causing a test failure
- (void)failWithException:(NSException *)exception { }

To not throw exceptions, but an easier solution is to use the GHAssertTrueNoThrow instead of GHAssertTrue macro.

answered Oct 09 '11 at 13:47

Ash Furrow

12,391
3
57
92

In my actual tests, I'm using OCHamcrest to write the assertions, because the GHAssert* syntax does not match the way I like to read & write assertions that well. Thanks for the `failWithException:` suggestion, but I think I'm happy with my own solution for now. – epologee Oct 09 '11 at 14:42

Why does a false assertion in async test in GHUnit crash the app instead of just failing the test?

4 Answers4

A solution.

Linked