How to use dispatch_queue_set_specific() and dispatch_get_specific()

Question

I'm having a hard time finding good examples on how to use these functions.

static void * kQueue1Key = "key1";
static void * kQueue2Key = "key2";

dispatch_queue_t queue1 = dispatch_queue_create("com.company.queue1", DISPATCH_QUEUE_SERIAL);
dispatch_queue_t queue2 = dispatch_queue_create("com.company.queue2", DISPATCH_QUEUE_SERIAL);

dispatch_queue_set_specific(queue1, kQueue1Key, (void *)kQueue1Key, NULL);
dispatch_queue_set_specific(queue2, kQueue2Key, (void *)kQueue2Key, NULL);

dispatch_sync(queue1, ^{
    if(dispatch_get_specific(kQueue1Key))
    {
        NSLog(@"I'm expecting this line to run (A)");

        dispatch_sync(queue2, ^{

            NSLog(@"I'm expecting this line to run (B)");

            if(dispatch_get_specific(kQueue2Key))
            {
                if(dispatch_get_specific(kQueue1Key))
                {
                    NSLog(@"I'm expecting this line to run (C)");
                }
                else
                {
                    [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (C)"];
                }
            }
            else
            {
                [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (B)"];
            }
        });
    }
    else
    {
        [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (A)"];
    }
});

Result

I'm expecting this line to run (A)
I'm expecting this line to run (B)
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Should not end up here (C)'

Is it expected behavior? If I were to dispatch_sync to queue1 since I'm not on the queue I would deadlock. What am I missing?

Are you, by chance, trying to implement recursive locks with `dispatch_sync`? Cause down this road lies eventual sadness (TLDR version: It can't work give the possibility of non-default queue targeting, and even if you assume your queues aren't cleverly targeted, the ancillary crap involved steals much (maybe all) of the performance benefit over standard recursive locking.) See also: http://stackoverflow.com/questions/19494167/how-to-implement-a-reentrant-locking-mechanism-in-objective-c-through-gcd/19495517#19495517 — ipmcc, Nov 07 '13 at 12:30
I thought `dispatch_get_specific` was the "new answer" to avoid mutex locks. If it's not, then what else does it offer? — hfossli, Nov 07 '13 at 15:21
Avoid mutex locks? How so? `dispatch_queue_set/get_specific` just provide arbitrary, associative storage hanging off a queue. `dispatch_get_specific` allows you to read that storage without having a reference to the current queue. There is some added functionality in that if you have Qb that targets Qa and you set_specific on Qa, and then get_specific from a block executing on Qb, it will walk the targeting chain and return you the value from Qa. — ipmcc, Nov 07 '13 at 15:34
Haha. Sounds very abstract. How can that be usefull? Is it GCD's superioer equivalent of `-[NSThread threadDictionary]`? — hfossli, Nov 07 '13 at 15:46
I don't think it's necessarily "superior" but there are certainly parallels between the two mechanisms, yes. The most frequent mention of `dispatch_queue_set/get_specific` is that it's the recommended mechanism that one can use to replace uses of the now-deprecated `dispatch_get_current_queue` function. — ipmcc, Nov 07 '13 at 16:07

ipmcc · Accepted Answer · 2013-11-09T23:17:06.633

Oh here, it popped into my head why you're getting what you're getting. Notes in line:

dispatch_sync(queue1, ^{

When you get to this point, the "current queue" is queue1

    if(dispatch_get_specific(kQueue1Key))

You're asking the current queue for the value it has for kQueue1Key, you set that earlier, so it gives it back to you.

    {
        NSLog(@"I'm expecting this line to run (A)");

        dispatch_sync(queue2, ^{

When you get to this point, the "current queue" is now queue2

            NSLog(@"I'm expecting this line to run (B)");

            if(dispatch_get_specific(kQueue2Key))

You're asking the current queue for the value it has for kQueue2Key, you set that earlier, so it gives it back to you.

            {

                if(dispatch_get_specific(kQueue1Key))

You're now asking the current queue for the value it has for kQueue1Key. Since the current queue is queue2 and you never set a value with kQueue1Key on queue2 you get back NULL.

                {
                    NSLog(@"I'm expecting this line to run (C)");
                }
                else
                {
                    [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (C)"];
                }

The misunderstanding here is that dispatch_get_specific doesn't traverse the stack of nested queues, it traverses the queue targeting lineage. For instance, if you did this instead,

static void * kQueue1Key = (void*)"key1";
static void * kQueue2Key = (void*)"key2";

dispatch_queue_t queue1 = dispatch_queue_create("com.company.queue1", DISPATCH_QUEUE_SERIAL);
dispatch_queue_t queue2 = dispatch_queue_create("com.company.queue2", DISPATCH_QUEUE_SERIAL);

dispatch_queue_set_specific(queue1, kQueue1Key, (void *)kQueue1Key, NULL);
dispatch_queue_set_specific(queue2, kQueue2Key, (void *)kQueue2Key, NULL);

// Set Queue2 to target Queue1
dispatch_set_target_queue(queue2, queue1);

dispatch_sync(queue2, ^{

    if(dispatch_get_specific(kQueue1Key))
    {
        NSLog(@"I'm expecting this line to run (A)");
    }
    else
    {
        [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (C)"];
    }

    if(dispatch_get_specific(kQueue2Key))
    {
        NSLog(@"I'm expecting this line to run (B)");
    }
    else
    {
        [NSException raise:NSInternalInconsistencyException format:@"Should not end up here (C)"];
    }
});

...the targeting relationship is the one that gets traversed, not the stack relationship. It would be nice if there were something that traversed the stack relationship, but I'm not aware of anything (that you wouldn't have to implement yourself).

This was really helpfull! Now I understand the API a lot better. And yep: you was correct about my assumption/misunderstanding. — hfossli, Nov 07 '13 at 18:16

score 3 · Answer 2 · answered Nov 07 '13 at 14:18

As mentioned in my comment, recursive locking using dispatch_sync is, in the general case, not possible due to the possibility of non-default queue targeting. For what it's worth, given/assuming default queue targeting, here is one possible approach:

#import <unordered_set>
#import <pthread.h>

static dispatch_once_t recursiveLockWithDispatchQueueTLSKeyOnceToken;
static pthread_key_t recursiveLockWithDispatchQueueTLSKey;
typedef std::unordered_multiset<const void*> RecursiveLockQueueBag;

static void freeRecursiveLockWithDispatchQueueTLSValue(void* tlsValue)
{
    RecursiveLockQueueBag* ms = reinterpret_cast<RecursiveLockQueueBag*>(tlsValue);
    if (ms) delete ms;
}

static inline BOOL queueStackCheck(dispatch_queue_t q, BOOL checkAndPushNotPop) // If yes, check and push if not on. If no, pop.
{
    dispatch_once(&recursiveLockWithDispatchQueueTLSKeyOnceToken, ^{
        pthread_key_create(&recursiveLockWithDispatchQueueTLSKey, freeRecursiveLockWithDispatchQueueTLSValue);
    });

    RecursiveLockQueueBag* ms = reinterpret_cast<RecursiveLockQueueBag*>(pthread_getspecific(recursiveLockWithDispatchQueueTLSKey));
    if (!ms)
    {
        ms = new RecursiveLockQueueBag();
        pthread_setspecific(recursiveLockWithDispatchQueueTLSKey, reinterpret_cast<const void*>(ms));
    }

    const void* const vpq = reinterpret_cast<const void*>((__bridge const void*)q);

    BOOL alreadyOn = NO;

    if (checkAndPushNotPop)
    {
        alreadyOn = (ms->count(vpq) > 0);
        if (!alreadyOn)
        {
            ms->insert(vpq);
        }
    }
    else
    {
        ms->erase(vpq);
    }
    return alreadyOn;
}

void dispatch_recursive_sync(dispatch_queue_t queue, dispatch_block_t block)
{
    if (queueStackCheck(queue, YES))
    {
        block();
    }
    else
    {
        @try
        {
            dispatch_sync(queue, block);
        }
        @finally
        {
            queueStackCheck(queue, NO);
        }
    }
}

@implementation MyAppDelegate

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
    dispatch_queue_t a = dispatch_queue_create("a", DISPATCH_QUEUE_SERIAL);
    dispatch_queue_t b = dispatch_queue_create("b", DISPATCH_QUEUE_SERIAL);
    dispatch_queue_t c = dispatch_queue_create("c", DISPATCH_QUEUE_SERIAL);
    //dispatch_set_target_queue(a, c);

    dispatch_recursive_sync(a, ^{
        dispatch_recursive_sync(b, ^{
            dispatch_recursive_sync(c, ^{
                dispatch_recursive_sync(a, ^{
                    dispatch_recursive_sync(b, ^{
                        dispatch_recursive_sync(c, ^{
                            dispatch_recursive_sync(a, ^{
                                NSLog(@"got there");
                            });
                        });
                    });
                });
            });
        });
    });


}

@end

This is the lowest-overhead implementation I could think of in a few minutes. I used C++ to avoid message sending overhead. It requires that all uses of the queue use this function. This can be useful when there's a private queue protecting internal state of an object (i.e. where the queue is private and therefore guaranteed not to be retargeted, and where you can easily ensure that all consumers of the queue use dispatch_recursive_sync.

Thanks for providing an answer even though this is not what I thought I wanted — hfossli, Nov 07 '13 at 14:55
Thank you for this. I packaged it in an SPM and Carthage-compatible project here: https://github.com/happn-tech/RecursiveSyncDispatch — Frizlab, Mar 31 '19 at 15:57
Also made the direct ObjectiveC++ implementation here (not packaged; it was just a test): https://gitlab.com/frizlab-demo-projects/test_recursive_dispatch — Frizlab, Jul 10 '19 at 21:37

How to use dispatch_queue_set_specific() and dispatch_get_specific()

2 Answers2

Linked