Linux IPC: Locking, but not file based locking

Question

I need a way to ensure only one python process is processing a directory.

The lock/semaphore should be local to the machine (linux operating system).

Networking or NFS is not involved.

I would like to avoid file based locks, since I don't know where I should put the lock file.

There are libraries which provide posix IPC at pypi.

Is there no way to use linux semaphores with python without a third party library?

The lock provided by multiprocessing.Lock does not help, since both python interpreter don't share one the same parent.

Threading is not involved. All processes have only one thread.

I am using Python 2.7 on linux.

How to to synchronize two python scripts on linux (without file based locking)?

Required feature: If one process dies, then the lock/semaphore should get released by the operating system.

Creation of a directory is atomic, so you could try and create a hidden directory called `.in_use` in the directory you are wanting to process and if you succeed, no-one else can succeed till you unlock by removing it.... — Mark Setchell, Nov 06 '17 at 13:18
@MarkSetchell I am searching a solution for the programming language python. — guettli, Nov 06 '17 at 13:49
@MarkSetchell creating a directory is not a solution. I updated the question: Required feature: If one process dies, then the lock/semaphore should get released by the operating system. — guettli, Nov 06 '17 at 13:50

score 2 · Accepted Answer · answered Nov 06 '17 at 16:10

2

flock the directory itself — then you never need worry about where to put the lock file:

import errno
import fcntl
import os
import sys

# This will work on Linux

dirfd = os.open(THE_DIRECTORY, os.O_RDONLY)         # FIXME: FD_CLOEXEC
try:
  fcntl.flock(dirfd, fcntl.LOCK_EX|fcntl.LOCK_NB)
except IOError as ex:
  if ex.errno != errno.EAGAIN:
    raise
  print "Somebody else is working here; quitting."  # FIXME: logging
  sys.exit(1)

do_the_work()
os.close(dirfd)

answered Nov 06 '17 at 16:10

pilcrow

56,591
13
94
135

OK, this works for this time, since in this case I have a directory. I am curious ... is there no way to do locking without a directory? – guettli Nov 07 '17 at 09:43

score 0 · Answer 2 · answered Nov 06 '17 at 16:10

I would like to avoid file based locks, since I don't know where I should put the lock file.

You can lock the existing file or directory (the one being processed).

Required feature: If one process dies, then the lock/semaphore should get released by the operating system.

That is exactly how file locks work.

Linux IPC: Locking, but not file based locking

2 Answers2