27
void WorkHandler::addWork(Work* w){
    printf("WorkHandler::insertWork Thread, insertWork locking \n");
    lock();
    printf("WorkHandler::insertWork Locked, and inserting into queue \n");
    m_workQueue.push(w);
    signal();
    unLock();
}

I followed a tutorial and I got this. I was wondering if it is ok to change the order of singal() and unLock() like this

void WorkHandler::addWork(Work* w){
    printf("WorkHandler::insertWork Thread, insertWork locking \n");
    lock();
    printf("WorkHandler::insertWork Locked, and inserting into queue \n");
    m_workQueue.push(w);
    unLock();
    signal();
}

If I can't do this, could you please give me details why I am not allowed to do this? Thanks in advance.

user800799
  • 2,883
  • 7
  • 31
  • 36
  • man page for pthread_cond_signal says you can signal whether or not you own the mutex (essentially anytime you are between lock and unlock) – Chris Jun 21 '11 at 00:41
  • possible duplicate of [pthread_cond_wait and mutex requirement](http://stackoverflow.com/questions/6312342/pthread-cond-wait-and-mutex-requirement) – Nemo Jun 21 '11 at 00:52

3 Answers3

36

First, there is no correctness issue here. Either order will work. Recall that whenever you use condition variables, you must loop on a predicate while waiting:

pthread_mutex_lock(mutex);
while (!predicate)
  pthread_cond_wait(cvar);
pthread_mutex_unlock(mutex);

By signalling after the unlock, you don't introduce any correctness issues; the thread is still guaranteed to wake up, and the worst case is another wakeup comes first - at which point it sees the predicate becomes true and proceeds.

However, there are two possible performance issues that can come up.

  • "Hurry up and wait". Basically, if you signal while the lock is held, the other thread still needs to wait until the mutex is available. Many pthreads implementations will, instead of waking up the other thread, simply move it to the wait queue of the mutex, saving an unnecessary wakeup->wait cycle. In some cases, however this is unimplemented or unavailable, leading to a potential spurious context switch or IPI.
  • Spurious wakeups. If you signal after the unlock, it's possible for another thread to issue another wakeup. Consider the following scenario:

    1. Thread A starts waiting for items to be added to a threadsafe queue.
    2. Thread B inserts an item on the queue. After unlocking the queue, but before it issues the signal, a context switch occurs.
    3. Thread C inserts an item on the queue, and issues the cvar signal.
    4. Thread A wakes up, and processes both items. It then goes back to waiting on the queue.
    5. Thread B resumes, and signals the cvar.
    6. Thread A wakes up, then immediately goes back to sleep, because the queue is empty.

    As you can see, this can introduce a spurious wakeup, which might waste some CPU time.

Personally, I don't think it's worth worrying too much about it either way. You don't often know offhand whether your implementation supports moving waiters from the condition variable to the mutex wait queue, which is the only real criterion you could use to decide which to use.

My gut feeling would be that, if I had to choose, signalling after the unlock is marginally less likely to introduce an inefficiency, as the inefficiency requires a three-thread race, rather than a two-thread race for the "hurry up and wait" condition. However, this is not really worth worrying about, unless benchmarks show too much context switch overhead or something.

bdonlan
  • 224,562
  • 31
  • 268
  • 324
  • 2
    perfect answer. this comprehends everything there is to know. – v.oddou Mar 24 '14 at 02:58
  • IPI? Inter-Path Interference? – user1284631 Oct 15 '14 at 12:05
  • @axeoth, Inter-Processor Interrupt. Basically an interrupt sent from one core to another. – bdonlan Dec 08 '14 at 06:57
  • I think signal first and then unlock will also cause Spurious wakeups due to mesa semantics. though it's less possible than the other. [Mesa and Hoare semantics](https://samuelsorial.tech/mesa-vs-hoare-semantics) – shino Oct 25 '21 at 08:49
3

This article is really worth reading towards your question:

Signal with mutexed or not?

Assuming you use same mutex with conditional variable to make condition change to be atomic. There are two cases and you should know their behavior:

  1. wait on signal (conditional var) while holding the mutex. The result is to let thread join the conditional var's queue and then go to sleep.
  2. signaled but without mutex. For this case, the thread won't sleep but block on it. (A mistake I made on this is that I thought it will sleep too. In this case, if producer signals and context switch happens right before it releases mutex, then all threads will wake up and know they can't lock the mutex, go to sleep forever. This is wrong because they won't sleep but wait and block).

Pthreads are implemented with wait-morphing, that is, instead of waking up threads upon signaling, it just transfer threads on conditional variable to the attached mutex queue. So signal while locking is more preferable without too much performance impact.

For signaling before unlocking mutex, it may causes spurious wake-up. If your code is not well designed to handle predicate changes made by spurious wake-up, you should choose signal while holding the lock.

user7610
  • 25,267
  • 15
  • 124
  • 150
Izana
  • 2,537
  • 27
  • 33
2

The answer to your question is "Yes". In fact, it's slightly preferable (as you've probably guessed) as it avoids the 'hurry up and wait' issue of waking up a thread to test a condition only to have it immediately block on the mutex it needs to acquire before testing the condition.

This answer is predicated on the guess that these things hold true:

  • lock is a thin wrapper for pthread_mutex_lock.
  • unLock is a thin wrapper for pthread_mutex_unlock.
  • signal is a thing wrapper for pthread_cond_signal.
  • The mutex your locking and unlocking is the one that your giving to pthread_cond_wait.
Omnifarious
  • 54,333
  • 19
  • 131
  • 194
  • Not true. The waiting thread cannot continue until it grabs the mutex, which cannot happen until the "signaller" unlocks it. The original order is actually better for performance. (See the accepted answer at the "duplicate" question for details.) – Nemo Jun 21 '11 at 00:54
  • @Nemo: I would like to see a performance analysis. `pthread_cond_wait` has to give up the mutex until it's signaled, then it has to immediately re-acquire the mutex. Being signaled while the mutex is held by someone else seems like it would force the signaled thread to wake up, try to grab the mutex and go to sleep again until the mutex was free. – Omnifarious Jun 21 '11 at 00:58
  • The point is that the "hurry up and wait" case -- where it has to re-test the condition -- can only happen in your proposed version (because the condition might become false between the time the signaler releases the mutex and when it signals the condition). With the standard (recommended) sequence, a good threading implementation will impose zero additional overhead (i.e., move waiters to an appropriate wait queue for the mutex). With your suggestion, there is always room for inefficiency... There is a reason every example in every reference does it the same way. – Nemo Jun 21 '11 at 01:05
  • @Nemo: That wasn't the 'hurry up and wait' case that I was talking about. I was talking about the one you say various implementations handle efficiently. I agree that signaling after releasing the mutex does open the possibility of the condition changing before the waiter can be woken up. I stand by my answer regardless. – Omnifarious Jun 21 '11 at 01:18
  • 1
    You are of course free to "stand by" your answer. But if you are going to recommend a sequence opposite to what _every other reference in the world recommends_ on the grounds of "performance", I think the burden is on you to provide the performance analysis. So I stand by my downvote. – Nemo Jun 21 '11 at 01:50
  • 1
    @Nemo, In some cases, moving waiters to the mutex's wait queue is not possible. This is the case on Linux when the condvar has its `pshared` attribute set, for example, as the mutex's address may be different or completely unavailable in the process signalling the condvar. Indeed, this is one of the motivating reasons for not making pshared default (the other being that a mm-private futex hash table reduces the occurance of hash collisions) – bdonlan Jun 21 '11 at 01:52
  • @bdonlan: That is good to know. But the answer to this question is still "Yes, it works, but it is not preferable in general". Sure, on _some_ implementations, assuming a _single_ waiter, and assuming `pshared` mutexes, there _might_ be a trivial performance advantage to the "reversed" sequence. In all other cases, the standard sequence is better. In my view, that still leaves this answer incorrect as currently phrased. – Nemo Jun 21 '11 at 01:59
  • 1
    @Nemo, yes, it's probably marginally better that way if you know that your pthreads implementation supports moving waiters. On the other hand, signalling after unlock is better (probably by a larger margin) if you know it doesn't support moving waiters. But either way it's fairly minor unless this is a very heavily trafficked cvar... – bdonlan Jun 21 '11 at 02:02
  • 1
    @bdonlan: But signaling after unlock introduces inefficiencies _regardless_ of the implementation (because one potential waiter can change the condition before another waiter receives the signal). Agree it is likely to be minor either way, but the standard sequence is the standard sequence for a good reason. – Nemo Jun 21 '11 at 02:08
  • @Nemo signalling after unlock introduces inefficiences which, although independent of implementation, require a race involving three threads in order to occur. So my gut feeling is it would occur less frequently than a "hurry up and wait" condition, on a system in which both are possible. But this is something you'd need to measure to be really sure. – bdonlan Jun 21 '11 at 02:10