Why doesn't multiprocessing Lock acquiring work?

Question

Tried 2 code examples from first answer here: Python sharing a lock between processes. Result is the same.

import multiprocessing
import time
from threading import Lock


def target(arg):
    if arg == 1:
        lock.acquire()
        time.sleep(1.1)
        print('hi')
        lock.release()
    elif arg == 2:
        while True:
            print('not locked')
            time.sleep(0.5)


def init(lock_: Lock):
    global lock
    lock = lock_


if __name__ == '__main__':
    lock_ = multiprocessing.Lock()
    with multiprocessing.Pool(initializer=init, initargs=[lock_], processes=2) as pool:
        pool.map(target, [1, 2])

Why does this code prints:

not locked
not locked
not locked
hi
not locked

instead

hi
not locked

Tim Peters · Accepted Answer · 2021-11-06T23:38:05.637

Well, call your worker processes "1" and "2". They both start. 2 prints "not locked", sleeps half a second, and loops around to print "not locked" again. But note that what 2 is printing has nothing do with whether lock is locked. Nothing in the code 2 executes even references lock, let alone synchronizes on lock. After another half second, 2 wakes up to print "not locked" for a third time, and goes to sleep again.

While that's going on, 1 starts, acquires the lock, sleeps for 1.1 seconds, and then prints "hi". It then releases the lock and ends. At the time 1 gets around to printing "hi", 2 has already printed "not locked" three times, and is about 0.1 seconds into its latest half-second sleep.

After "hi" is printed, 2 will continue printing "not locked" about twice per second forever more.

So the code appears to be doing what it was told to do.

What I can't guess, though, is how you expected to see "hi" first and then "not locked". That would require some kind of timing miracle, where 2 didn't start executing at all before 1 had been running for over 1.1 seconds. Not impossible, but extremely unlikely.

Changes

Here's one way to get the output you want, although I'm making many guesses about your intent.

If you don't want 2 to start before 1 ends, then you have to force that. One way is to have 2 begin by acquiring lock at the start of what it does. That also requires guaranteeing that lock is in the acquired state before any worker begins.

So acquire it before map() is called. Then there's no point left to having 1 acquire it at all - 1 can just start at once, and release it when it ends, so that 2 can proceed.

There are few changes to the code, but I'll paste all of it in here for convenience:

import multiprocessing
import time
from threading import Lock

def target(arg):
    if arg == 1:
        time.sleep(1.1)
        print('hi')
        lock.release()
    elif arg == 2:
        lock.acquire()
        print('not locked')
        time.sleep(0.5)

def init(lock_: Lock):
    global lock
    lock = lock_


if __name__ == '__main__':
    lock_ = multiprocessing.Lock()
    lock_.acquire()
    with multiprocessing.Pool(initializer=init, initargs=[lock_], processes=2) as pool:
        pool.map(target, [1, 2])

Thanks! I finally figured out how lock works. I thought that when you call lock.acquire(), the rest of the threads are simply blocked. — Den Avrondo, Nov 07 '21 at 05:24
Nope - not even close ;-) The only effect a successful `.acquire()` has is to cause all future attempts to `.acquire()` to block, until (if ever) `.release()` is called. If `.release()` is called, one outstanding (blocked) `.acquire()` attempt will succeed (if at least one such exists), but which one is not defined. — Tim Peters, Nov 07 '21 at 05:40

Why doesn't multiprocessing Lock acquiring work?

1 Answers1

Changes