Prevent a second process from listening to the same pipe in Python

Question

I have a process that connects to a pipe with Python 2.7's multiprocessing.Listener() and waits for a message with recv(). I run it various on Windows 7 and Ubuntu 11.

On Windows, the pipe is called \\.\pipe\some_unique_id. On Ubuntu, the pipe is called /temp/some_unique_id. Other than that, the code is the same.

All works well, until, in an unrelated bug, monit starts a SECOND copy of the same program. It tries to listen to the exact same pipe.

I had naively^* expected that the second connection attempt would fail, leaving the first connection unscathed.

Instead, I find the behaviour is officially undefined.

Note that data in a pipe may become corrupted if two processes (or threads) try to read from or write to the same end of the pipe at the same time.

On Ubuntu, the earlier copies seem to be ignored, and are left without any messages, while the latest version wins.

On Windows, there is some more complex behaviour. Sometimes the original pipe raises an EOFError exception on the recv() call. Sometimes, both listeners are allowed to co-exist and each message is distributed arbitrarily.

Is there a way to open a pipe exclusively, so the second process cannot open the pipe while the first process hasn't closed it or exited?

^* I could have sworn I manually tested this exact scenario, but clearly I did not.

Other SO questions I looked at:

• several TCP-servers on the same port - I don't (knowngly) set SO_REUSEADDR
• Can two applications listen to the same port?
• accept() with sockets shared between multiple processes (based on Apache preforking) - there's no forking involved.

Answer 1

Named pipes have the same access symantics as regular files. Any process with read or write permission can open the pipe for reading or writing.

If you had a way to guarantee that the two instances of the Python script were invoked by processes with differing UID's or GID's, then you can implement unique access control using file permissions.

If both instances of the script have the same UID and GID, you can try file locking implemented in Skip Montanaro's FileLock hosted on github. YMMV.

A simpler way to implement this might be to create a lock file in /var/lock that contains the PID of the process creating the lock file and then check for the existence of the lock file before opening the pipe. This scheme is used by most long-running daemons but has problems when the processes that create the lock files terminate in situations that prevent them from removing the lock file.

You could also try a Python System V semaphore to prevent synchronous access.