36

I'm working on a GUI application in WxPython, and I am not sure how I can ensure that only one copy of my application is running at any given time on the machine. Due to the nature of the application, running more than once doesn't make any sense, and will fail quickly. Under Win32, I can simply make a named mutex and check that at startup. Unfortunately, I don't know of any facilities in Linux that can do this.

I'm looking for something that will automatically be released should the application crash unexpectedly. I don't want to have to burden my users with having to manually delete lock files because I crashed.

Asclepius
  • 57,944
  • 17
  • 167
  • 143
Matt Green
  • 2,032
  • 2
  • 22
  • 36

12 Answers12

63

The Right Thing is advisory locking using flock(LOCK_EX); in Python, this is found in the fcntl module.

Unlike pidfiles, these locks are always automatically released when your process dies for any reason, have no race conditions exist relating to file deletion (as the file doesn't need to be deleted to release the lock), and there's no chance of a different process inheriting the PID and thus appearing to validate a stale lock.

If you want unclean shutdown detection, you can write a marker (such as your PID, for traditionalists) into the file after grabbing the lock, and then truncate the file to 0-byte status before a clean shutdown (while the lock is being held); thus, if the lock is not held and the file is non-empty, an unclean shutdown is indicated.

Asclepius
  • 57,944
  • 17
  • 167
  • 143
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • 1
    This is the correct method to lock, however its also good to write a PID file that is cleaned up on normal exit, as well as to create an entry in /var/lock/subsys (if it exists). This allows your program to realize if its re-starting from a crash, among other things. So, doing both helps. – Tim Post Feb 18 '09 at 06:07
  • @tinkertim - Not a bad suggestion, though it makes sense to flock() the pidfile rather than having more than one. – Charles Duffy Feb 18 '09 at 17:10
  • are the locks released on system crashes / someone turning off the power to your machine? – jedierikb Mar 23 '11 at 12:23
  • 2
    @jedierikb POSIX advisory locks only last as long as the file descriptors do. Yes, they're correctly released on hard shutdown and the like. – Charles Duffy Mar 23 '11 at 22:51
  • 1
    @Kaz, eh? These patterns solve completely different problems. `pthread_mutex_t` is appropriate when you're protecting in-memory content, sure -- but if you're protecting content on a filesystem, the other processes holding the lock may not even live on the same host. And if there are recovery operations to be taken when taking ownership of on-disk data in an unknown state, it's wise to *always* perform those when granted the lock -- for all you know, the system may have suffered a hard shutdown (ie. power loss) prior. – Charles Duffy Sep 08 '17 at 21:00
32

Complete locking solution using the fcntl module:

import fcntl
pid_file = 'program.pid'
fp = open(pid_file, 'w')
try:
    fcntl.lockf(fp, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
    # another instance is running
    sys.exit(1)
Asclepius
  • 57,944
  • 17
  • 167
  • 143
zgoda
  • 12,775
  • 4
  • 37
  • 46
  • 2
    Assuming the lock file is the same for all users, as it ought to be for the lock to be useful, this can create a write permissions problem. I have described and addressed this problem in an answer. – Asclepius Feb 22 '13 at 04:20
  • I don't know why but this does not work in Python 3.4.1. Two instances run without any errors. – boreq May 26 '14 at 15:52
  • @boreq, could you be a little more explicit about exactly how you're testing (filesystem in use, expected behavior, actual behavior, etc)? – Charles Duffy Aug 27 '14 at 00:01
  • @boreq, ...for instance, if one were *deleting* the pidfiles on exit, that would definitely break it in the manner described. – Charles Duffy Jul 22 '16 at 17:03
25

There are several common techniques including using semaphores. The one I see used most often is to create a "pid lock file" on startup that contains the pid of the running process. If the file already exists when the program starts up, open it up and grab the pid inside, check to see if a process with that pid is running, if it is check the cmdline value in /proc/pid to see if it is an instance of your program, if it is then quit, otherwise overwrite the file with your pid. The usual name for the pid file is application_name.pid.

Robert Gamble
  • 106,424
  • 25
  • 145
  • 137
  • Out of curiosity, doesn't simply opening the file for exclusive access do the work of a mutex? – Menkboy Oct 21 '08 at 02:27
  • Menkboy, if the file is properly closed in the event of a crash, then I think that will work perfectly and simplify things further. Thank you. – Matt Green Oct 21 '08 at 02:36
  • 1
    This is the first description of pidfiles on SO I've seen that explains all the steps needed to make sure they work properly. Also, the last two steps can be done at once - checking /proc/pid/cmdline will fail if 'pid' isn't a running process, since the directory won't exist. – Branan Oct 21 '08 at 17:25
  • 14
    This appears to have a race condition? First, suppose an instance of my program dies, leaving behind the stale PID file. Then I start up two new instances of my program at the same time. The two new instances might both decide that the PID file is stale, and then one after another they will overwrite the PID file. This is not a problem for traditional UNIX daemons, which are only started at boot and by explicit administrator command-line choice. But it might be a problem for other programs. – user9876 Nov 25 '09 at 12:15
  • Note that procfs is quite specifically a Linux thing -- this code will not work under MacOS X, the BSDs, or traditional Unixen. To be sure, this question is tagged `linux`, but it's still worth taking portability concerns into account. – Charles Duffy Jun 19 '14 at 20:35
  • 1
    @user9876 This has to be implemented by means of a complete removal of the stale `PID` file, and an atomic creation via `O_CREAT` and `O_EXCL`. The condition for being the single instance has to be "I successfully created the file, with `O_EXCL`." Then you're good --- unless `/var/run` is NFS mounted, oops. Due to NFS, this has to be a directory: as a rule of thumb, `mkdir` is atomic on NFS; file creation may not be. – Kaz Jun 20 '17 at 04:39
  • 2
    @Kaz I think that just moves the race to the delete. Suppose: Process A creates the file then crashes, leaving the file behind. Processes B and C then start, and both check the file and see it contains the stale PID, so they both decide to delete-and-recreate. Process B deletes the file and atomically recreates it. Then process C deletes the file and atomically recreates it. Processes B and C both think they have the exclusive lock. Fundamentally, handling stale locks without kernel support is hard - that's why you should use flock() instead. – user9876 Sep 08 '17 at 17:18
  • I think this works because PID's are usually assigned sequentially - https://superuser.com/questions/135007/how-are-pids-generated - but if they were assigned randomly, one day it might fail. https://stackoverflow.com/questions/11323410/linux-pid-recycling – cardamom Jan 22 '19 at 12:53
8

wxWidgets offers a wxSingleInstanceChecker class for this purpose: wxPython doc, or wxWidgets doc. The wxWidgets doc has sample code in C++, but the python equivalent should be something like this (untested):

  name = "MyApp-%s" % wx.GetUserId()
  checker = wx.SingleInstanceChecker(name)
  if checker.IsAnotherRunning():
      return False
Brian Victor
  • 387
  • 1
  • 4
  • 12
6

This builds upon the answer by user zgoda. It mainly addresses a tricky concern having to do with write access to the lock file. In particular, if the lock file was first created by root, another user foo can then no successfully longer attempt to rewrite this file due to an absence of write permissions for user foo. The obvious solution seems to be to create the file with write permissions for everyone. This solution also builds upon a different answer by me, having to do creating a file with such custom permissions. This concern is important in the real world where your program may be run by any user including root.

import fcntl, os, stat, tempfile

app_name = 'myapp'  # <-- Customize this value

# Establish lock file settings
lf_name = '.{}.lock'.format(app_name)
lf_path = os.path.join(tempfile.gettempdir(), lf_name)
lf_flags = os.O_WRONLY | os.O_CREAT
lf_mode = stat.S_IWUSR | stat.S_IWGRP | stat.S_IWOTH  # This is 0o222, i.e. 146

# Create lock file
# Regarding umask, see https://stackoverflow.com/a/15015748/832230
umask_original = os.umask(0)
try:
    lf_fd = os.open(lf_path, lf_flags, lf_mode)
finally:
    os.umask(umask_original)

# Try locking the file
try:
    fcntl.lockf(lf_fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
    msg = ('Error: {} may already be running. Only one instance of it '
           'can run at a time.'
           ).format('appname')
    exit(msg)

A limitation of the above code is that if the lock file already existed with unexpected permissions, those permissions will not be corrected.

I would've liked to use /var/run/<appname>/ as the directory for the lock file, but creating this directory requires root permissions. You can make your own decision for which directory to use.

Note that there is no need to open a file handle to the lock file.

Asclepius
  • 57,944
  • 17
  • 167
  • 143
4

Here's the TCP port-based solution:

# Use a listening socket as a mutex against multiple invocations
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('127.0.0.1', 5080))
s.listen(1)
shuckc
  • 2,766
  • 1
  • 22
  • 17
1

The set of functions defined in semaphore.h -- sem_open(), sem_trywait(), etc -- are the POSIX equivalent, I believe.

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Menkboy
  • 1,595
  • 10
  • 12
1

Look for a python module that interfaces to SYSV semaphores on unix. The semaphores have a SEM_UNDO flag which will cause the resources held by the a process to be released if the process crashes.

Otherwise as Bernard suggested, you can use

import os
os.getpid()

And write it to /var/run/application_name.pid. When the process starts, it should check if the pid in /var/run/application_name.pid is listed in the ps table and quit if it is, otherwise write its own pid into /var/run/application_name.pid. In the following var_run_pid is the pid you read from /var/run/application_name.pid

cmd = "ps -p %s -o comm=" % var_run_pid
app_name = os.popen(cmd).read().strip()
if len(app_name) > 0:
    Already running
Matthew Smith
  • 6,165
  • 6
  • 34
  • 35
  • 1
    +1 for suggesting SYSV semaphores, -1 for suggesting calling `ps` rather than something more efficient (say, `kill -0` -- via the signal call rather than the command if one prefers to avoid an extra fork/exec) and suggesting pidfiles without advisory locking (ie. flock) to avoid race conditions and PID collisions. – Charles Duffy May 16 '10 at 03:20
0

If you create a lock file and put the pid in it, you can check your process id against it and tell if you crashed, no?

I haven't done this personally, so take with appropriate amounts of salt. :p

Bernard
  • 45,296
  • 18
  • 54
  • 69
  • 2
    Using the PID file is common, but is not without issues. For one, there can be race conditions. Second, it may not get cleaned up if the app is killed. – Asclepius Feb 20 '13 at 21:02
0

Can you use the 'pidof' utility? If your app is running, pidof will write the Process ID of your app to stdout. If not, it will print a newline (LF) and return an error code.

Example (from bash, for simplicity):

linux# pidof myapp
8947
linux# pidof nonexistent_app

linux#
Adam Liss
  • 47,594
  • 12
  • 108
  • 150
  • This is neat, less code required than the lockfile approach. Downside is it isn't exactly atomic, but for single instance detection, that may not be necessary. – Matt Green Oct 21 '08 at 02:23
  • This won't work if you're running "pidof app" from inside the app – Matthew Smith Oct 21 '08 at 02:39
  • 4
    It also won't work if there is a different running program with the same name, or if another user is also running the program. – CesarB Oct 21 '08 at 13:10
0

By far the most common method is to drop a file into /var/run/ called [application].pid which contains only the PID of the running process, or parent process. As an alternative, you can create a named pipe in the same directory to be able to send messages to the active process, e.g. to open a new file.

user29832
  • 9
  • 1
  • Using the PID file is common, but is not without issues. For one, there can be race conditions. Second, it may not get cleaned up if the app is killed. – Asclepius Feb 20 '13 at 20:58
0

I've made a basic framework for running these kinds of applications when you want to be able to pass the command line arguments of subsequent attempted instances to the first one. An instance will start listening on a predefined port if it does not find an instance already listening there. If an instance already exists, it sends its command line arguments over the socket and exits.

code w/ explanation

Zach Dwiel
  • 529
  • 1
  • 4
  • 18