Running python script with cron only if not running

Question

I need to run a python script (job.py) every minute. This script must not be started if it is already running. Its execution time can be between 10 seconds and several hours.

So I put into my crontab:

* * * * * root cd /home/lorenzo/cron && python -u job.py 1>> /var/log/job/log 2>> /var/log/job/err

To avoid starting the script when it is already running, I use flock().

This is the script (job.py):

import fcntl
import time
import sys

def doIncrediblyImportantThings ():
    for i in range (100):
        sys.stdout.write ('[%s] %d.\n' % (time.strftime ('%c'), i) )
        time.sleep (1)

if __name__ == '__main__':
    f = open ('lock', 'w')
    try: fcntl.lockf (f, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except:
        sys.stderr.write ('[%s] Script already running.\n' % time.strftime ('%c') )
        sys.exit (-1)
    doIncrediblyImportantThings ()

This approach seems to work.

Is there anything I am missing? Are there any troubles I can run into using this approach?

Are there more advised or "proper" ways of achieving this behaviour?

I thank you for any suggestion.

score 15 · Accepted Answer · answered May 27 '11 at 01:28

The only suggestion I would make is to make your exception handling a little more specific. You don't want to accidentally delete the fcntl import one day and hide the NameError that results. Always try to catch the most specific exception you want to handle. In this case, I suggest something like:

import errno

try:
    fcntl.lock(...)
except IOError, e:
    if e.errno == errno.EAGAIN:
        sys.stderr.write(...)
        sys.exit(-1)
    raise

This way, any other cause of the lock being unobtainable shows up (probably in your email since you're using cron) and you can decide if it's something for an administrator to look at, another case for the program to handle, or something else.

For those using Python 3: Since Python 3.3, fcntl uses OSError instead of IOError — R.M., Aug 22 '22 at 22:26

score 3 · Answer 2 · answered May 27 '11 at 00:30

3

You're in trouble when the machine reboots or freezes with the script running (and thus an active lock). Simple way to counter this is to use the @reboot cron timestamp to run rm /path/to/lock.

answered May 27 '11 at 00:30

Mel

6,077
1
15
12

Thank you very much. I will take this into consideration. Are file locks persistent through reboots? Depends this on the file system I am using (actually ext4)? – Hyperboreus May 27 '11 at 00:40
2

File locks are not persistent across reboots. They aren't even persistent across processes restarting, which is why you don't have to release the lock in your code - it is released when the process terminates. – Jean-Paul Calderone May 27 '11 at 01:24
@Jean-Paul This means I do not have worry about reboots and freezes as Mel states? – Hyperboreus May 27 '11 at 02:33

score 2 · Answer 3 · answered Jan 28 '15 at 06:48

I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI.

Install with: pip install quicklock

Using it is extremely simple:

[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
    raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!

Take a look: https://pypi.python.org/pypi/quicklock

Running python script with cron only if not running

3 Answers3

Linked