subprocess.popen detached from master (Linux)

Question

I am trying to open a subprocess but have it be detached from the parent script that called it. Right now if I call subprocess.popen and the parent script crashes the subprocess dies as well.

I know there are a couple of options for windows but I have not found anything for *nix.

I also don't need to call this using subprocess. All I need is to be able to cal another process detached and get the pid.

Possibly related: http://stackoverflow.com/questions/5772873/python-spawn-off-a-child-subprocess-detach-and-exit — NightShadeQueen, Jul 17 '15 at 18:30
I saw that but i am not trying to detach and exit. I need the parent script to continue to run, but if it were to crash I don't want the subprocess to stop as well. — korki696, Jul 17 '15 at 18:33
so dont call `os._exit()` in the parent process .. (where it checks the PID in the active state recipe from the linked answer) http://code.activestate.com/recipes/278731-creating-a-daemon-the-python-way/ — Joran Beasley, Jul 17 '15 at 18:50

score 3 · Answer 1 · answered Jul 17 '15 at 19:18

With linux, it's no issue at all. Just Popen(). For example, here is a little dying_demon.py

#!/usr/bin/python -u
from time import sleep
from subprocess import Popen
print Popen(["python", "-u", "child.py"]).pid
i = 0
while True:
    i += 1
    print "demon: %d" % i
    sleep(1)
    if i == 3:
        i = hurz # exception

spinning off a child.py

#!/usr/bin/python -u
from time import sleep
i = 0
while True:
    i += 1
    print "child: %d" % i
    sleep(1)
    if i == 20:
        break

The child continues to count (to the console), while the demon is dying by exception.

You can get away with this if your parent python process crashes. But incorporate this kind of code in something higher level that handles the exception, and you will be leaving zombie processes unless you close it out with wait() or similar. — Joshua Clayton, Jul 23 '18 at 22:25

score 1 · Answer 2 · answered Jul 17 '15 at 18:59

1

I think this should do the trick: https://www.python.org/dev/peps/pep-3143/#reference-implementation

You can create daemon which will call your subprocess, passing detach_process=True.

answered Jul 17 '15 at 18:59

Filip Malczak

3,124
24
44

score 1 · Answer 3 · answered Nov 06 '22 at 08:20

This might do what you want:

def cmd_detach(*command, **kwargs) -> subprocess.CompletedProcess:
    # https://stackoverflow.com/questions/62521658/python-subprocess-detach-a-process
    # if using with ffmpeg remember to run it with `-nostdin`
    stdout = os.open(os.devnull, os.O_WRONLY)
    stderr = os.open(os.devnull, os.O_WRONLY)
    stdin = os.open(os.devnull, os.O_RDONLY)

    command = conform(command)
    if command[0] in ["fish", "bash"]:
        import shlex
        command = command[0:2] + [shlex.join(command[2:])]
    subprocess.Popen(command, stdin=stdin, stdout=stdout, stderr=stderr, close_fds=True, start_new_session=True, **kwargs)
    return subprocess.CompletedProcess(command, 0, "Detached command is async")

On Windows you might need

CREATE_NEW_PROCESS_GROUP = 0x00000200
DETACHED_PROCESS = 0x00000008
creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP

instead of start_new_session=True

score 0 · Answer 4 · answered Jul 17 '15 at 20:24

I managed to get it working by doing the following using python-daemon:

process = subprocess.Popen(["python", "-u", "Child.py"])
    time.sleep(2)
    process.kill()

Then in Child.py:

with daemon.DaemonContext():
    print("Child Started")
    time.sleep(30)
    print "Done"
    exit()

I do process.kill() because otherwise it creates a defunct python process. The main problem I have now is that the PID that popen returns does not match the final pid of the process. I can get by this by adding a function in Child.py to update a database with the pid.

Let me know if there is something that I am missing or if this is an ok method of doing this.

score 0 · Answer 5 · answered Jul 17 '15 at 20:30

0

fork the subprocs using the NOHUP option

answered Jul 17 '15 at 20:30

jobeard

129
6

I think that doesn't work unless you use `shell=True`, which can be a burden if not otherwise needed. – Nick T Apr 18 '16 at 12:41

subprocess.popen detached from master (Linux)

5 Answers5

Linked