Error with XCTestExpectation: API violation - multiple calls made to -[XCTestExpectation fulfill]

Question

I'm using XCTestExpectations in Xcode 6 (Beta 5) for asynchronous testing. All my asynchronous tests pass individually every time I run them. However, when I try to run my entire suite, some tests do not pass, and the app crashes.

The error I get is says API violation - multiple calls made to -[XCTestExpectation fulfill]. Indeed, this is not true within a single method; my general format for my tests is shown below:

- (void) someTest {
    /* Declare Expectation */
    XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
    [MyClass loginOnServerWithEmail:@"example@email.com" andPassword:@"asdfasdf" onSuccess:^void(User *user) {
        /* Make some assertions here about the object that was given. */

        /* Fulfill the expectation */
        [expectation fulfill];
    }];

    [self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
        /* Error handling here */
    }];
}

Again, these tests do pass when run individually, and they are actually making network requests (working exactly as intended), but together, the collection of tests fail to run.

I was able to have a look at this post here, but was unable to get the solution to work for me.

Additionally, I'm running OSX Mavericks and using Xcode 6 (Beta 5).

Answer 1

XCTestExpectation Error

API violation - multiple calls made to -[XCTestExpectation fulfill]

I got the same error when set expectation.expectedFulfillmentCount.

Solution:

expectation.assertForOverFulfill = false

If expectation.assertForOverFulfill = true and fulfill() is called when it is already fulfilled then it throws an exception

Answer 2

I don't think using __weak or __block is a good approach. I have written many unit tests using XCTestExpectation for awhile and never had this problem until now. I finally found out that real cause of the problem which potentially may cause bugs in my app. The root cause of my problem is that startAsynchronousTaskWithDuration calls the completionHandler multiple time. After I fix it the API violation went away!

[self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
    XCTAssertNotNil(result);
    XCTAssertNil(error);
    [expectation fulfill];
}];

Although it took me a few hours to fix my unit tests but I came to appreciate the API violation error which will help me avoid future runtime problem in my app.

Answer 3

Had the same issue on Xcode 12.5 / Swift 5.

Check if you're not overriding setUp and tearDown 'class' methods.

It should be:

override func setUp()
override func tearDown()

Instead of:

override class func setUp()
override class func tearDown()

Answer 4

Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'API violation - multiple calls made to -[XCTestExpectation fulfill] for whatever.'

I get the error above with the following code:

func testMultipleWaits() {
    let exp = expectation(description: "whatever")
    for _ in 0...10 {
        DispatchQueue.main.async {
            exp.fulfill()
        }
        wait(for: [exp], timeout: 1)
    }
}

Basically a given expectation can be fulfilled once, it can also be waited once.

Meaning the following change would still not fix it, because then you've still waited multiple times for it. It would give you the following error.

failed: caught "NSInternalInconsistencyException", "API violation - expectations can only be waited on once, whatever have already been waited on"

func testMultipleWaits() {
    let exp = expectation(description: "whatever")
    for i in 0...10 {
        DispatchQueue.main.async {
            if i == 6 {
                exp.fulfill()
            }
        }
        wait(for: [exp], timeout: 1)
    }
}

What makes it confusing is that while the above change still crashes, the test does get finished and gives you a failure of "Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "whatever".'

It's misleading because you may spend time fixing the test, all while you should be spending time fixing the crash which is the root cause

---

The fix here is set the expectation inside the for-loop. This way the expectation is fulfilled and waited against once per expectation.

func testMultipleWaits() {        
    for _ in 0...10 {
        let exp = expectation(description: "whatever")
        DispatchQueue.main.async {
            exp.fulfill()
        }
        wait(for: [exp], timeout: 1)
    }
}

Answer 5

Even though this question is quite old, I don't think it has a proper explanation of the root problems. This is surely hitting one of the problems you can get with XCTestExpectation. For a full deep explanation, go to Jeremy W. Sherman post where he explained XCTestExpectation gotchas perfectly.

Summary:

• Always assign your expectations to a weak reference, and then bail in your callback if it’s nil.
• In the rare case where you expect your callback to be triggered more than once, you can avoid fulfilling by annihilating your weak reference after fulfilling it and then ignoring future calls. More likely, you know how many times you should be called, and you’ll want to fulfill the promise only on the last call. But the workaround is there if you need it.
• If you’re already working with a promise-based API, you can skip XCTestExpectation and use whatever wait-and-see API is provided by that promise instead of XCTest’s own. This has the added advantage of linearizing your test code by eliminating the need to handle the delivered value in the closure (or manually shuttle it out to assert against after the XCTest wait has finished).

Code examples properly avoiding errors and crashes:

func testPreparedForNotWaitingLongEnough() {
    weak var promiseToCallBack = expectationWithDescription("calls back")
    after(seconds: callBackDelay) { () -> Void in
        guard let promise = promiseToCallBack else {
            print("too late, buckaroo")
            return
        }

        print("I knew you'd call!")
        promise.fulfill()
    }

    waitForExpectationsWithTimeout(callBackDelay / 2) { error in
        print("Aww, we timed out: \(error)")
    }
}

func testSafelyDoubleTheFulfillment() {
    weak var promiseToCallBack = expectationWithDescription("calls back")
    let callBackDelay: NSTimeInterval = 1

    twiceAfter(seconds: callBackDelay) {
        guard let promise = promiseToCallBack else {
            print("once was enough, thanks!")
            return
        }

        promise.fulfill()
        promiseToCallBack = nil
    }

    let afterCallBack = 2 * callBackDelay
    waitForExpectationsWithTimeout(afterCallBack, handler: nil)
}

Answer 6

I think it's likely that you have a retain cycle issue somewhere preventing to release your object which is calling the block in which the expectation is fulfill multiple time.

Otherwise, if it's an expected behavior that your expectation is called multiple time, I've write a little extension allowing to specify an expectation count:

import XCTest

extension XCTestExpectation {
    private class IntWrapper {
        let value :Int!
        init?(value:Int?) {
            self.value = value
            if (value == nil) {
                return nil
            }
        }
    }

    private struct AssociatedKey {
        static var expectationCountKey = ""
    }

    var expectationCount:Int? {
        get {
            return objc_getAssociatedObject(self, &AssociatedKey.expectationCountKey) as? Int
        }
        set {
            objc_setAssociatedObject(self, &AssociatedKey.expectationCountKey, newValue, .OBJC_ASSOCIATION_RETAIN)
        }
    }

    func decrementalFulfill() {
        guard let expectationCount = self.expectationCount else {
            fulfill()
            return
        }
        self.expectationCount = expectationCount - 1
        if self.expectationCount <= 0 {
            fulfill()
        }
    }
}

full code (with test :) here: https://gist.github.com/huguesbr/7d110bffd043e4d11f2886693c680b06

Answer 7

Folks who stuck with the API violation error, possibly the self.expectation() is not your friend. Instead, you can try a regular init:

let expectation = XCTestExpectation(description: "")

Check this answer for the details.

Answer 8

I was not sure why it was calling fulfilled multiple times, I resolved this by doing something like:

let tokenReceived = expectation(description: "Token Received")
var fulfilled = false
mymanager.fetchToken(task: "token", callbackBlock: {token, error in
            DispatchQueue.main.async {
                tokenString = token
                debugPrint("Token found:\(token)")
                if(!fulfilled)
                {
                    fulfilled.toggle()
                    tokenReceived.fulfill()
                }
            }
        })

Answer 9

My problem was also threading issue with my code. I did not remove a value in a dictionary of completion handlers. That value was then reused by my next test calling the completion handler again (and also fulfilling the expectation again), which caused the error.

Please Note: it is in your best interest to find / fix these and do not assume an issue with the expectation or Xcode.

Answer 10

Often this is caused if you forget to call return after calling the completionBlock.

func load(complete: ((Result)->Void)) {
   urlSession.dataTask(request) { (error, response, data) in 
      if let error = error {
         complete(.failure(error) // 1st time calls complete(_) 
         // missing return statement here
      }
      complete(.success(data!) // 2nd time calls complete(_)
   }.resume()
}

Answer 11

Here is probably the answer you are looking for :

XCTestExpectation: how to avoid calling the fulfill method after the wait context has ended?

It at least fixed the issue for me.

Answer 12

Try declaring your expectationWithDescription as weak and unwrap the optional "expected" variable.

 weak var asyncExpectation = expectationWithDescription("expectation")

 check for options in the block.
 if let asyncExpectation = asyncExpectation{
    asyncExpectation.fulfill()
 }

This avoids deallocation of asyncExpectation variable and calling your expectation on nil.