57

I'm working on a nifty little function:

def startProcess(name, path):
    """
    Starts a process in the background and writes a PID file

    returns integer: pid
    """

    # Check if the process is already running
    status, pid = processStatus(name)

    if status == RUNNING:
        raise AlreadyStartedError(pid)

    # Start process
    process = subprocess.Popen(path + ' > /dev/null 2> /dev/null &', shell=True)

    # Write PID file
    pidfilename = os.path.join(PIDPATH, name + '.pid')
    pidfile = open(pidfilename, 'w')
    pidfile.write(str(process.pid))
    pidfile.close()

    return process.pid

The problem is that process.pid isn't the correct PID. It seems it's always 1 lower than the correct PID. For instance, it says the process started at 31729, but ps says it's running at 31730. Every time I've tried it's off by 1. I'm guessing the PID it returns is the PID of the current process, not the started one, and the new process gets the 'next' pid which is 1 higher. If this is the case, I can't just rely on returning process.pid + 1 since I have no guarantee that it'll always be correct.

Why doesn't process.pid return the PID of the new process, and how can I achieve the behaviour I'm after?

Hubro
  • 56,214
  • 69
  • 228
  • 381
  • 1
    Try removing `shell=True` and your `>` stuff. – Blender Nov 03 '11 at 03:26
  • 1
    i have the same problem and i found out the hard way, it's not always PID+1, it was at first, but now it's giving me +2 – ierdna Feb 10 '14 at 19:59
  • @Hubro I'd like to use this nifty function in some of my own code! Would you mind posting your code for `processStatus()`? – Jordan Jul 25 '19 at 22:08
  • @Jordan It's been way too long, sorry. I have no idea what I was even writing this for. – Hubro Jul 26 '19 at 00:36

1 Answers1

55

From the documentation at http://docs.python.org/library/subprocess.html:

Popen.pid The process ID of the child process.

Note that if you set the shell argument to True, this is the process ID of the spawned shell.

If shell is false, it should behave as you expect, I think.

If you were relying on shell being True for resolving executable paths using the PATH environment variable, you can accomplish the same thing using shutil.which instead, then pass the absolute path to Popen instead. (As an aside, if you are using Python 3.5 or newer, you should be using subprocess.run rather than Popen.

Community
  • 1
  • 1
krousey
  • 1,728
  • 15
  • 22
  • 1
    I needed `shell=True` for the relative path to python to work. I guess I'll replace it with the absolute path and set `shell` to false. Thanks for the information! – Hubro Nov 03 '11 at 03:37
  • 1
    Passing the path through os.path.abspath first should fix that for you. http://docs.python.org/library/os.path.html – krousey Nov 03 '11 at 03:39
  • Nah, running `os.path.abspath('python')` just returns my working directory plus 'python', e.g. '/root/python' – Hubro Nov 03 '11 at 03:47
  • 1
    Ah I see, you were relying on the shell's PATH variable. Another trick you can use is executing `/usr/bin/env python` instead. I've seen it mostly used in shell scripts, but may help you here. – krousey Nov 03 '11 at 03:58
  • Ah yes, that would probably work, but wouldn't that cause the same issue? Wouldn't `pid` return the PID of `env`? Or does `env` perhaps use `exec`? – Hubro May 19 '16 at 06:38
  • 1
    So I guess you really want to do $PATH resolution in Python then. http://stackoverflow.com/questions/377017/test-if-executable-exists-in-python has an answer that should work for that. – krousey May 25 '16 at 16:02
  • >> >>> import os >>> import subprocess >>> from subprocess import Popen >>> import requests >>> from time import sleep >>> env = dict(os.environ) >>> env['JAVA_OPTS'] = '8000 7' >>> jarFile = '/Path to jar file/myfile.jar' >>> process = subprocess.Popen(['java', '-jar', jarFile], env=env, shell=True) >>> pid = Popen.pid Traceback (most recent call last): File "", line 1, in pid = Popen.pid AttributeError: type object 'Popen' has no attribute 'pid' – SimonM Aug 09 '16 at 13:23
  • SimonM: you need to get process.pid, not Popen.pid – Marc Jan 13 '17 at 20:08