non-blocking lock with 'with' statement

Question

As far as I know, the following code will be blocked if lock is already acquired by another thread.

It seems that non-blocking can be implemented by lock.acquire(0), but instead I have to use try-finally block instead with block.

lock = threading.Lock()

def func():
 with lock:
  # do something...

Is there any method to implement non-blocking lock acquisition?

score 13 · Answer 1 · answered Oct 26 '16 at 08:21

13

@contextmanager
def nonblocking(lock):
    locked = lock.acquire(False)
    try:
        yield locked
    finally:
        if locked:
            lock.release()

lock = threading.Lock()
with nonblocking(lock) as locked:
    if locked:
        do_stuff()

answered Oct 26 '16 at 08:21

Николай Коновалов

131
1
5

2

Welcome to Stack Overflow! While this code may answer the question, providing additional context regarding why and/or how this code answers the question improves its long-term value. Code-only answers are discouraged. – Ajean Oct 26 '16 at 18:58

score 10 · Answer 2 · answered Jul 19 '15 at 13:53

10

Is there any method to implement non-blocking lock acquisition?

Yes. Just raise an exception if the lock can't be acquired immediately. Something like:

@contextlib.contextmanager
def non_blocking_lock(lock=threading.Lock()):
    if not lock.acquire(blocking=False):
        raise WouldBlockError
    try:
        yield lock
    finally:
        lock.release()

Usage:

with non_blocking_lock():
    # run with the lock acquired

answered Jul 19 '15 at 13:53

jfs

399,953
195
994
1,670

1

Thanks for answering. Anyway I have to use `lock.acquire` function. – JaeJun LEE Jul 19 '15 at 17:17
I don't understand the point of the default argument. The lock would be created inside the context manager, and will be provided back to the user already locked - but of course no other thread knows about this lock. In what scenario could this be useful? – max May 10 '17 at 21:26
@max: It is not how default arguments work in Python. Understanding ["Least Astonishment" and the Mutable Default Argument](http://stackoverflow.com/q/1132941/4279) might help. – jfs May 10 '17 at 21:28
1

@J.F.Sebastian Ahh right! I know how mutable default arguments work, but I didn't realize that feature can be used for good rather than for evil :) Cool, so you get this one effectively global lock instance, "hidden" by the context manager definition, and you can implicitly rely on it everywhere by just using this context manager without arguments. That said, it seems a bit dangerous, since an accidental omission of the argument would cause a very subtle bug that will be super hard to detect. – max May 10 '17 at 21:51
1

@max if you want your own lock; you pass your own lock. Otherwise, you shouldn't pass any arguments to the context manager. Yes, it is very easy to introduce bugs in a multithreaded code. – jfs May 10 '17 at 22:00
One problem with this solution - the `finally` block would be called even if the lock was not acquired, so the lock would be immediately released by the thread which failed to acquire it, and not by the tread which owns it - usually not what you want. – bavaza May 31 '22 at 10:12
1

@bavaza: false. (`raise` is outside try/finally) – jfs Jun 01 '22 at 06:09

Mayor Mayer · Answer 3 · 2021-04-14T12:28:33.797

If you need a context manager that acquires a lock in a non-blocking manner, but still retries until the lock can finally be acquired, you could do like this:

@contextlib.contextmanager
def non_blocking_lock(lock : threading.Lock):
    # Waits as long as the lock can not be acquired, but releases the GIL in the meanwhile
    while not lock.acquire(blocking=False):
        pass

    try:
        yield   # Lock has been successfully acquired
    finally:
        lock.release()

It can be used exactly like the normal lock context manager:

class TestClass:
    def __init__(self):
         self._lock = threading.Lock()
    
    def method(self):
         with non_blocking_lock(self._lock):
         # do something that should be only done from one thread at once

... with the difference, that the lock is non-blocking and doesn't hold the GIL until the lock is released. I used it to fix some deadlocks.

The difference to the other solutions is, that the code eventually gets executed and the context manager does not simply return a false or throw an exception when the lock couldn't be acquired.

Correct me if you see any caveats with this solution.

non-blocking lock with 'with' statement

3 Answers3