fcntl.flock - how to implement a timeout?

Question

I am using python 2.7

I want to create a wrapper function around fcntl.flock() that will timeout after a set interval:

wrapper_function(timeout):

I've tried calling on another thread and using thread.join(timeout) but it seems that fcntl.flock() continues blocking:

def GetLock(self, timeout):
    """Returns true if lock is aquired, false if lock is already in use"""
    self.__lock_file = open('proc_lock', 'w')

    def GetLockOrTimeOut():
        print 'ProcessLock: Acquiring Lock'            
        fcntl.flock(self.__lock_file.fileno(), fcntl.LOCK_EX)
        print 'ProcessLock: Lock Acquired'

    thread = threading.Thread(target=GetLockOrTimeOut)
    thread.start()
    thread.join(timeout)

    if thread.isAlive():
        print 'GetLock timed out'
        return False
    else:
        return True

I've looked into solutions for terminating threads, the most popular solution seems to be sub-classing threading.thread and adding a feature to raise an exception in the thread. However, I came across a link that says this method will not work with native calls, which I am pretty sure fcntl.flock() is calling a native function. Suggestions?

Context: I am using a file-lock to create a single instance application but I don't want a second instance of the application to sit around and hang until the first instance terminates.

Answer 1

Timeouts for system calls are done with signals. Most blocking system calls return with EINTR when a signal happens, so you can use alarm to implement timeouts.

Here's a context manager that works with most system calls, causing IOError to be raised from a blocking system call if it takes too long.

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        pass

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except IOError, e:
        if e.errno != errno.EINTR:
            raise e
        print "Lock timed out"

Answer 2

I'm sure there are several ways, but how about using a non-blocking lock? After some n attempts, give up and exit?

To use non-blocking lock, include the fcntl.LOCK_NB flag, as in:

fcntl.flock(self.__lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)

Answer 3

For Python 3.5+, Glenn Maynard's solution no longer works because of PEP-475. This is a modified version:

import signal, errno
from contextlib import contextmanager
import fcntl

@contextmanager
def timeout(seconds):
    def timeout_handler(signum, frame):
        # Now that flock retries automatically when interrupted, we need
        # an exception to stop it
        # This exception will propagate on the main thread, make sure you're calling flock there
        raise InterruptedError

    original_handler = signal.signal(signal.SIGALRM, timeout_handler)

    try:
        signal.alarm(seconds)
        yield
    finally:
        signal.alarm(0)
        signal.signal(signal.SIGALRM, original_handler)

with timeout(1):
    f = open("test.lck", "w")
    try:
        fcntl.flock(f.fileno(), fcntl.LOCK_EX)
    except InterruptedError:
        # Catch the exception raised by the handler
        # If we weren't raising an exception, flock would automatically retry on signals
        print("Lock timed out")

Answer 4

I'm a fan of shelling out to flock here, since attempting to do a blocking lock with a timeout requires changes to global state, which makes it harder to reason about your program, especially if threading is involved.

You could fork off a subprocess and implement the alarm as above, or you could just exec http://man7.org/linux/man-pages/man1/flock.1.html

import subprocess
def flock_with_timeout(fd, timeout, shared=True):
    rc = subprocess.call(['flock', '--shared' if shared else '--exclusive', '--timeout', str(timeout), str(fd)])
    if rc != 0:
        raise Exception('Failed to take lock')

If you have a new enough version of flock you can use -E to specify a different exit code for the command otherwise succeeding, but failed to take the lock after a timeout, so you can know whether the command failed for some other reason instead.

Answer 5

As a complement to @Richard Maw answer above https://stackoverflow.com/a/32580233/17454091 (Don't have enough reputation to post a comment).

In Python 3.2 and newer, for fds to be available in sub-processes one must also provide pass_fds argument.

Complete solution ends up as:

import subprocess
def flock_with_timeout(fd, timeout, shared=True):
    rc = subprocess.call(['flock',
                          '--shared' if shared else '--exclusive',
                          '--timeout', str(timeout),
                          str(fd)],
                         pass_fds=[fd])
    if rc != 0:
        raise Exception('Failed to take lock')