Killing a subprocess including its children from python

Question

I'm using the subprocess module on python 2.5 to spawn a java program (the selenium server, to be precise) as follows:

import os
import subprocess

display = 0
log_file_path = "/tmp/selenium_log.txt"
selenium_port = 4455
selenium_folder_path = "/wherever/selenium/lies"

env = os.environ
env["DISPLAY"] = ":%d.0" % display
command = ["java", 
           "-server",
           "-jar", 
           'selenium-server.jar',
           "-port %d" % selenium_port]
log = open(log_file_path, 'a')
comm = ' '.join(command)
selenium_server_process = subprocess.Popen(comm,
                                           cwd=selenium_folder_path,
                                           stdout=log,
                                           stderr=log,
                                           env=env,
                                           shell=True)

This process is supposed to get killed once the automated tests are finished. I'm using os.kill to do this:

os.killpg(selenium_server_process.pid, signal.SIGTERM)
selenium_server_process.wait()

This does not work. The reason is that the shell subprocess spawns another process for java, and the pid of that process is unknown to my python code. I've tried killing the process group with os.killpg, but that kills also the python process which runs this code in the first place. Setting shell to false, thus avoiding java to run inside a shell environment, is also out of the question, due to other reasons.

How can I kill the shell and any other processes generated by it?

related: [How to terminate a python subprocess launched with shell=True](http://stackoverflow.com/q/4789837/4279) — jfs, Mar 22 '14 at 19:53

score 34 · Answer 1 · edited Dec 08 '15 at 14:00

34

To handle the general problem:

p=subprocess.Popen(your_command, preexec_fn=os.setsid)
os.killpg(os.getpgid(p.pid), signal.SIGTERM)

setsid will run the program in a new session, thus assigning a new process group to it and its children. calling os.killpg on it thus won't bring down your own python process also.

edited Dec 08 '15 at 14:00

DaveS

491
3
11

answered Mar 22 '14 at 19:42

berdario

1,851
18
29

No need to use `setsid`; you can [call `os.setsid` in Python](http://stackoverflow.com/a/4791612/4279) – jfs Mar 22 '14 at 19:54
No, you cannot... that will change the session of the process itself, if what you're after is killing only the children it isn't what you want – berdario Mar 22 '14 at 21:56
reread the title of the question: *"Killing a subprocess including its children from python"* – jfs Mar 23 '14 at 01:00
fyi, preexec_fn is executed *after* fork (in the child process) – jfs Mar 23 '14 at 01:51
I've read the title, have you? Thomas' answer didn't deal with killing the children of the subprocess (btw yes, prexec_fn is interesting, thanks) – berdario Mar 23 '14 at 13:48
the title says: process + children. You say: "only children". – jfs Mar 23 '14 at 13:57
1

the title says "subprocess" + children, I say "children" (meaning, subprocess+children), you say "the calling process itself + children (subprocess and its children)" – berdario Mar 23 '14 at 14:00
Just tried, seems no need `preexec_fn=os.setsid`. My python is 3.6.8 – TangHongWan Jan 22 '21 at 01:19
If you don't `preexec_fn=os.setsid `, your Python script is going to also kill the program that started the program that started the children. (i.e. the Python script itself in my short snippet) – berdario Jan 27 '21 at 08:02

score 4 · Answer 2 · answered Apr 14 '10 at 16:02

The obvious solution in this case is to not involve the shell:

import os
import subprocess

display = 0
log_file_path = "/tmp/selenium_log.txt"
selenium_port = 4455
selenium_folder_path = "/wherever/selenium/lies"

env = os.environ
env["DISPLAY"] = ":%d.0" % display
command = ["java", 
           "-server",
           "-jar", 
           'selenium-server.jar',
           "-port",
           str(selenium_port)]
log = open(log_file_path, 'a')
selenium_server_process = subprocess.Popen(command,
                                           cwd=selenium_folder_path,
                                           stdout=log,
                                           stderr=subprocess.STDOUT,
                                           env=env)

This will make the process be the Java process directly. Keep in mind that it may still spawn processes that are not part of the process group, so os.killpg may still not know about killing them.

If you have a reason to invoke the shell (the above code does not, and there are few things you cannot do without the shell, but suppose you do), you would have to make the shell pass you the pid of the process it started somehow. Doing this is not straightforward, and rather situational.

Or, if you do involve the shell for any reason (let's say you want expansion/substitution/whatever), use "exec" at the beginning to avoid the shell forking. (Specifically, just adding "exec" at the beginning of "command" definition would have fixed your problem.) — moshez, Apr 14 '10 at 18:36
Do you actually have a *reason* for involving the shell, though? There very rarely is. — Thomas Wouters, Apr 15 '10 at 16:40

Killing a subprocess including its children from python

2 Answers2

Linked

Related