How does a serial queue/private dispatch queue know when a task is complete?

Question

(Perhaps answered by How does a serial dispatch queue guarantee resource protection? but I don't understand how)

Question

How does gcd know when an asynchronous task (e.g. network task) is finished? Should I be using dispatch_retain and dispatch_release for this purpose? Update: I cannot call either of these methods with ARC... What do?

Details

I am interacting with a 3rd party library that does a lot of network access. I have created a wrapper via a small class that basically offers all the methods i need from the 3rd party class, but wraps the calls in dispatch_async(serialQueue) { () -> Void in (where serialQueue is a member of my wrapper class).

I am trying to ensure that each call to the underlying library finishes before the next begins (somehow that's not already implemented in the library).

Paulw11 · Accepted Answer · 2017-02-10T02:20:58.513

The serialisation of work on a serial dispatch queue is at the unit of work that is directly submitted to the queue. Once execution reaches the end of the submitted closure (or it returns) then the next unit of work on the queue can be executed.

Importantly, any other asynchronous tasks that may have been started by the closure may still be running (or may not have even started running yet), but they are not considered.

For example, for the following code:

dispatch_async(serialQueue) {
    print("Start")
    dispatch_async(backgroundQueue) {
       functionThatTakes10Seconds()
       print("10 seconds later")
    }
    print("Done 1st")
}

dispatch_async(serialQueue) {
    print("Start")
    dispatch_async(backgroundQueue) {
       functionThatTakes10Seconds()
       print("10 seconds later")
    }
    print("Done 2nd")
}

The output would be something like:

Start

Done 1st

Start

Done 2nd

10 seconds later

10 seconds later

Note that the first 10 second task hasn't completed before the second serial task is dispatched. Now, compare:

dispatch_async(serialQueue) {
    print("Start")
    dispatch_sync(backgroundQueue) {
       functionThatTakes10Seconds()
       print("10 seconds later")
    }
    print("Done 1st")
}

dispatch_async(serialQueue) {
    print("Start")
    dispatch_sync(backgroundQueue) {
       functionThatTakes10Seconds()
       print("10 seconds later")
    }
    print("Done 2nd")
}

The output would be something like:

Start

10 seconds later

Done 1st

Start

10 seconds later

Done 2nd

Note that this time because the 10 second task was dispatched synchronously the serial queue was blocked and the second task didn't start until the first had completed.

In your case, there is a very good chance that the operations you are wrapping are going to dispatch asynchronous tasks themselves (since that is the nature of network operations), so a serial dispatch queue on its own is not enough.

You can use a DispatchGroup to block your serial dispatch queue.

dispatch_async(serialQueue) {
    let dg = dispatch_group_create()
    dispatch_group_enter(dg)
    print("Start")
    dispatch_async(backgroundQueue) {
       functionThatTakes10Seconds()
       print("10 seconds later")
       dispatch_group_leave(dg)
    }
    dispatch_group_wait(dg)
    print("Done")
}

This will output

Start

10 seconds later

Done

The dg.wait() blocks the serial queue until the number of dg.leave calls matches the number of dg.enter calls. If you use this technique then you need to be careful to ensure that all possible completion paths for your wrapped operation call dg.leave. There are also variations on dg.wait() that take a timeout parameter.

this is FAN-TASTIC! question: if DispatchGroup is an unresolved identifier, does that mean I'm on Swift 2? 1? (I know i'm not on 3) Should I probably use dispatch_group_create, dispatch_group_enter, and dispatch_group_leave as in (e.g.) http://commandshift.co.uk/blog/2014/03/19/using-dispatch-groups-to-wait-for-multiple-web-services/ ? — Michael, Feb 10 '17 at 02:16
That's right; I default to Swift 3 :) I should probably clean up my answer, since at the moment it is a mixture of 2 and 3. — Paulw11, Feb 10 '17 at 02:18
do you think the dispatch_group_wait is necessary? like if I were going to have 2 of the blocks like you have at the very end of your answer, wouldn't the dispatch_group_enter and dispatch_group_leave calls implement serialization already? (and the output then be the same as that from your second block?) — Michael, Feb 10 '17 at 19:54
Nope, `dispatch_group_enter` doesn't block; you can call it as many times as you like. It just keeps increasing rhendispstch_group "count". `dispatch_group_leave` decreases that count and `dispatch_group_wait` blocks until the count is 0. You can also use `dispatch_group_notify` to submit a block to be executed when're count becomes 0. This is used when you want to execute some code when a number of asynchronous tasks that may be running in parallel have completed. — Paulw11, Feb 10 '17 at 21:09

score 2 · Answer 2 · answered Feb 12 '17 at 03:32

As mentioned before, DispatchGroup is a very good mechanism for that.

You can use it for synchronous tasks:

let group = DispatchGroup()
DispatchQueue.global().async(group: group) {
   syncTask()
}

group.notify(queue: .main) {
    // done
}

It is better to use notify than wait, as wait does block the current thread, so it is safe on non-main threads.

You can also use it to perform async tasks:

let group = DispatchGroup()
group.enter()
asyncTask {
   group.leave()
}

group.notify(queue: .main) {
    // done
}

Or you can even perform any number of parallel tasks of any synchronicity:

let group = DispatchGroup()

group.enter()
asyncTask1 {
   group.leave()
}

group.enter() //other way of doing a task with synchronous API
DispatchQueue.global().async {
   syncTask1()
   group.leave()
}

group.enter()
asyncTask2 {
   group.leave()
}

DispatchQueue.global().async(group: group) {
   syncTask2()
}

group.notify(queue: .main) {
    // runs when all tasks are done
}

It is important to note a few things.

Always check if your asynchronous functions call the completion callback, sometimes third party libraries forget about that, or cases when your self is weak and nobody bothered to check if the body got evaluated when self is nil. If you don't check it then you can potentially hang and never get the notify callback.
Remember to perform all the needed group.enter() and group.async(group: group) calls before you call the group.notify. Otherwise you can get a race condition, and the group.notify block can fire, before you actually finish your tasks.

BAD EXAMPLE

let group = DispatchGroup()

DispatchQueue.global().async {
   group.enter()
   syncTask1()
   group.leave()
}

group.notify(queue: .main) { 
    // Can run before syncTask1 completes - DON'T DO THIS
}

shallowThought · Answer 3 · 2017-02-11T18:08:24.937

0

The answer to the question in your questions body:

I am trying to ensure that each call to the underlying library finishes before the next begins

A serial queue does guarantee that the tasks are progressed in the order you add them to the queue.

I do not really understand the question in the title though:

How does a serial queue ... know when a task is complete?

edited Feb 11 '17 at 18:08

answered Feb 09 '17 at 20:07

shallowThought

19,212
9
65
112

How does a serial queue/private dispatch queue know when a task is complete?

3 Answers3

Linked