11

I am trying to run a Python script from another Python script, and getting its pid so I can kill it later.

I tried subprocess.Popen() with argument shell=True', but thepidattribute returns thepid` of the parent script, so when I try to kill the subprocess, it kills the parent.

Here is my code:

proc = subprocess.Popen(" python ./script.py", shell=True)
pid_ = proc.pid
.
.
.
# later in my code

os.system('kill -9 %s'%pid_)

#IT KILLS THE PARENT :(
alex
  • 6,818
  • 9
  • 52
  • 103
farhawa
  • 10,120
  • 16
  • 49
  • 91

2 Answers2

16

shell=True starts a new shell process. proc.pid is the pid of that shell process. kill -9 kills the shell process making the grandchild python process into an orphan.

If the grandchild python script can spawn its own child processes and you want to kill the whole process tree then see How to terminate a python subprocess launched with shell=True:

#!/usr/bin/env python
import os
import signal
import subprocess

proc = subprocess.Popen("python script.py", shell=True, preexec_fn=os.setsid) 
# ...
os.killpg(proc.pid, signal.SIGTERM)

If script.py does not spawn any processes then use @icktoofay suggestion: drop shell=True, use a list argument, and call proc.terminate() or proc.kill() -- the latter always works eventually:

#!/usr/bin/env python
import subprocess

proc = subprocess.Popen(["python", "script.py"]) 
# ...
proc.terminate()

If you want to run your parent script from a different directory; you might need get_script_dir() function.

Consider importing the python module and running its functions, using its object (perhaps via multiprocessing) instead of running it as a script. Here's code example that demonstrates get_script_dir() and multiprocessing usage.

Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • 1
    It should be `os.killpg(os.getpgid(proc.pid), signal.SIGTERM)` instead of `os.killpg(proc.pid, signal.SIGTERM)`. – PJ_Finnegan Jun 21 '18 at 16:02
  • @PJ_Finnegan: it works as is. `os.getpgid(proc.pid) == proc.pid` here. – jfs Jun 21 '18 at 16:09
  • +jfs true, but in the cited example and in some code I've tested, it also worked without using `setsid()` in the Popen and using `killpg`. I think that `Popen` automatically assigns a new group id to the top shell. Plus, `preexec_fn` is not thread safe. Check this: https://stackoverflow.com/q/42257512/1141215 – PJ_Finnegan Jun 22 '18 at 21:35
  • @PJ_Finnegan: 1- if you remove setsid(); the code breaks 2- there is no `start_new_session` in Python 2. Look at the shebang (it is `python`, not `python3`) 3- rule of thumb: unless the code is explicitly called "thread-safe"; it is not. You should not assume that any code is thread-safe unless you see the explicit claim to the contrary. – jfs Jun 22 '18 at 21:45
4

So run it directly without a shell:

proc = subprocess.Popen(['python', './script.py'])

By the way, you may want to consider changing the hardcoded 'python' to sys.executable. Also, you can use proc.kill() to kill the process rather than extracting the PID and using that; furthermore, even if you did need to kill by PID, you could use os.kill to kill the process rather than spawning another command.

icktoofay
  • 126,289
  • 21
  • 250
  • 231
  • @farhawa: Can you update your question to clarify what’s happening? I understood you as saying that there’s a Python interpreter spawning a shell spawning another Python interpreter, and the kill kills the shell rather than the desired subordinate interpreter. – icktoofay Jun 25 '15 at 02:39