"Fire and forget" a process from a Python script

Question

How do I start a process (another Python script, for example) from a Python script so the "child" process is completely detached from the "parent", so the parent can a) continue on its merry way without waiting for child to finish and b) can be terminated without terminating the child process?

Parent:

import os

print "Parent started"
os.system("./child.py")
print "Parent finished"

Child:

import time

print "Child started"
time.sleep(10)
print "Child finished"

Running parent.py prints:

Parent started
Child started
Child finished
Parent finished

What I want it to print:

Parent started
Child started
Parent finished
(seconds later)
Child finished

Look at `subprocess` module, and specifically at `subprocess.Popen(...).pid` — sberry, Jul 30 '13 at 02:19
@sberry: Nice, looks like just using `subprocess.Popen("./child.py")` does exactly what I need although is't not clear at all from the docs. If you add your comment as an answer I'll be happy to accept it, thanks. — Sergey, Jul 30 '13 at 02:35
Depending on the platform, you could probably add an `&` at the end of the shell command and it would also work. — kevinsa5, Jul 30 '13 at 04:05
@kevinsa5: yep, that works even better - I just found that with Popen(), if the parent process is killed the child seems to be killed too. — Sergey, Jul 30 '13 at 04:24
Note that adding `&` [works on Linux](https://stackoverflow.com/a/13338895), but [does not work on Windows](https://stackoverflow.com/a/8055390): you would have to use [start](https://superuser.com/q/467012) instead. — djvg, Nov 17 '21 at 14:16

RubenLaguna · Answer 1 · 2014-06-11T19:40:01.817

12

Since you mentioned os.system, I think it's worth to mention that you should have used os.spawn* with mode P_NOWAIT to achieve the "forget" part.

But subprocess module provides replacements for os.system, os,spawn*,etc so you should use that instead like so

import subprocess
p = subprocess.Popen("./child.py")
print "pid = ", p.pid

See Replacing os.spawn with subprocess.Popen

As I explained in the comments both processes parent.py and child.py are still on the same process group and therefore the terminal will forward signals (like Ctrl-C) to all process in the foreground process group so both will get killed when you Ctrl-C. So if you don't want that you can force child.py to be in a new process group with the following:

#!/usr/bin/env python
import subprocess
import time
import os
p = subprocess.Popen("./child.py", preexec_fn=os.setsid)
print "pid = ", p.pid
time.sleep(30) # Ctrl-C at this point will not kill child.py
print "parent exit"

edited Jun 11 '14 at 19:40

answered Jun 10 '14 at 06:31

RubenLaguna

21,435
13
113
151

Hey, thanks for answering this old question. There's a problem with `subprocess.Popen` though: if the parent process is terminated - the child is getting terminated too. – Sergey Jun 10 '14 at 20:05
What do you mean? I tried this piece of code and the child didn't get terminated at least if parent dies of natural causes. Ctrl-C (all signals) are different the terminal receives the signal and forwards it on to any process in the foreground process group. The process group here includes parent.py and child.py so both get killed. I will try to provide the specific Popen sequence to create child in a new process group. – RubenLaguna Jun 11 '14 at 19:28
1

Looks like `preexec_fn` is [POSIX only](https://docs.python.org/3/library/subprocess.html#subprocess.Popen). Here's [how to do it on Windows](https://stackoverflow.com/a/14797454). – djvg Nov 17 '21 at 14:30

nehem · Answer 2 · 2019-01-21T07:01:03.633

Using asyncio you can write a simple decorator as @background

import asyncio
import time

def background(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, *kwargs)

    return wrapped

@background
def foo():
    time.sleep(1)
    print("foo() completed")

print("Hello")
foo()
print("I didn't wait for foo()")

Produces

>>> Hello
>>> I didn't wait for foo()
>>> foo() completed

score 1 · Accepted Answer · edited Nov 17 '21 at 15:48

Answering my own question: I ended up simply using os.system with & at the end of command as suggested by @kevinsa. This allows the parent process to be terminated without the child being terminated.

Here's some code:

child.py

#!/usr/bin/python

import time
print "Child started"
time.sleep(10)
print "Child finished"

parent.py, using subprocess.Popen:

#!/usr/bin/python

import subprocess
import time

print "Parent started"
subprocess.Popen("./child.py")
print "(child started, sleeping)"

time.sleep(5)

print "Parent finished"

Output:

$ ./parent.py
Parent started
(child started, sleeping)
Child started
^CTraceback (most recent call last):
Traceback (most recent call last):
  File "./child.py", line 5, in <module>
  File "./parent.py", line 13, in <module>
        time.sleep(10)
time.sleep(5)
KeyboardInterrupt
KeyboardInterrupt

note how the child never finishes if the parent is interrupted with Ctrl-C

parent.py, using os.system and &

#!/usr/bin/python

import os
import time

print "Parent started"
os.system("./child.py &")
print "(child started, sleeping)"

time.sleep(5)

print "Parent finished"

Output:

$ ./parent.py
Parent started
(child started, sleeping)
Child started
^CTraceback (most recent call last):
  File "./parent.py", line 12, in <module>
    time.sleep(5)
KeyboardInterrupt

$ Child finished

Note how the child lives beyond the Ctrl-C.

[subprocess vs os.system](https://stackoverflow.com/q/44730935). — djvg, Nov 17 '21 at 15:41

"Fire and forget" a process from a Python script

3 Answers3