How to terminate a python subprocess launched with shell=True

Question

I'm launching a subprocess with the following command:

p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)

However, when I try to kill using:

p.terminate()

or

p.kill()

The command keeps running in the background, so I was wondering how can I actually terminate the process.

Note that when I run the command with:

p = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)

It does terminate successfully when issuing the p.terminate().

Answer 1

Use a process group so as to enable sending a signal to all the process in the groups. For that, you should attach a session id to the parent process of the spawned/child processes, which is a shell in your case. This will make it the group leader of the processes. So now, when a signal is sent to the process group leader, it's transmitted to all of the child processes of this group.

Here's the code:

import os
import signal
import subprocess

# The os.setsid() is passed in the argument preexec_fn so
# it's run after the fork() and before  exec() to run the shell.
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, 
                       shell=True, preexec_fn=os.setsid) 

os.killpg(os.getpgid(pro.pid), signal.SIGTERM)  # Send the signal to all the process groups

Answer 2

p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
p.kill()

p.kill() ends up killing the shell process and cmd is still running.

I found a convenient fix this by:

p = subprocess.Popen("exec " + cmd, stdout=subprocess.PIPE, shell=True)

This will cause cmd to inherit the shell process, instead of having the shell launch a child process, which does not get killed. p.pid will be the id of your cmd process then.

p.kill() should work.

I don't know what effect this will have on your pipe though.

Answer 3

If you can use psutil, then this works perfectly:

import subprocess

import psutil


def kill(proc_pid):
    process = psutil.Process(proc_pid)
    for proc in process.children(recursive=True):
        proc.kill()
    process.kill()


proc = subprocess.Popen(["infinite_app", "param"], shell=True)
try:
    proc.wait(timeout=3)
except subprocess.TimeoutExpired:
    kill(proc.pid)

Answer 4

I could do it using

from subprocess import Popen

process = Popen(command, shell=True)
Popen("TASKKILL /F /PID {pid} /T".format(pid=process.pid))

it killed the cmd.exe and the program that i gave the command for.

(On Windows)

Answer 5

When shell=True the shell is the child process, and the commands are its children. So any SIGTERM or SIGKILL will kill the shell but not its child processes, and I don't remember a good way to do it. The best way I can think of is to use shell=False, otherwise when you kill the parent shell process, it will leave a defunct shell process.

Answer 6

None of these answers worked for me so Im leaving the code that did work. In my case even after killing the process with .kill() and getting a .poll() return code the process didn't terminate.

Following the subprocess.Popen documentation:

"...in order to cleanup properly a well-behaved application should kill the child process and finish communication..."

proc = subprocess.Popen(...)
try:
    outs, errs = proc.communicate(timeout=15)
except TimeoutExpired:
    proc.kill()
    outs, errs = proc.communicate()

In my case I was missing the proc.communicate() after calling proc.kill(). This cleans the process stdin, stdout ... and does terminate the process.

Answer 7

As Sai said, the shell is the child, so signals are intercepted by it -- best way I've found is to use shell=False and use shlex to split the command line:

if isinstance(command, unicode):
    cmd = command.encode('utf8')
args = shlex.split(cmd)

p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

Then p.kill() and p.terminate() should work how you expect.

Answer 8

Send the signal to all the processes in group

    self.proc = Popen(commands, 
            stdout=PIPE, 
            stderr=STDOUT, 
            universal_newlines=True, 
            preexec_fn=os.setsid)

    os.killpg(os.getpgid(self.proc.pid), signal.SIGHUP)
    os.killpg(os.getpgid(self.proc.pid), signal.SIGTERM)

Answer 9

There is a very simple way for Python 3.5 or + (Actually tested on Python 3.8)

import subprocess, signal, time
p = subprocess.Popen(['cmd'], shell=True)
time.sleep(5) #Wait 5 secs before killing
p.send_signal(signal.CTRL_C_EVENT)

Then, your code may crash at some point if you have a keyboard input detection, or sth like this. In this case, on the line of code/function where the error is given, just use:

try:
    FailingCode #here goes the code which is raising KeyboardInterrupt
except KeyboardInterrupt:
    pass

What this code is doing is just sending a "CTRL+C" signal to the running process, what will cause the process to get killed.

Answer 10

Solution that worked for me

if os.name == 'nt':  # windows
    subprocess.Popen("TASKKILL /F /PID {pid} /T".format(pid=process.pid))
else:
    os.kill(process.pid, signal.SIGTERM)

Answer 11

Full blown solution that will kill running process (including subtree) on timeout reached or specific conditions via a callback function. Works both on windows & Linux, from Python 2.7 up to 3.10 as of this writing.

Install with pip install command_runner

Example for timeout:

from command_runner import command_runner

# Kills ping after 2 seconds
exit_code, output = command_runner('ping 127.0.0.1', shell=True, timeout=2)

Example for specific condition: Here we'll stop ping if current system time seconds digit is > 5

from time import time
from command_runner import command_runner

def my_condition():
    # Arbitrary condition for demo
    return True if int(str(int(time()))[-1]) > 5

# Calls my_condition() every second (check_interval) and kills ping if my_condition() returns True
exit_code, output = command_runner('ping 127.0.0.1', shell=True, stop_on=my_condition, check_interval=1)