nested subprocess not stopping

Question

I have 3 files:

sleeper.py

import subprocess

print('start sleeper')
subprocess.run(['sleep', '10'])
print('end sleeper')

waker.py

import subprocess

print('The waker begins')
try:
    subprocess.run(['python3', 'sleeper.py'], timeout=5)
except subprocess.TimeoutExpired:
    pass
print('The waker ends')

awake.py

import subprocess

print('Awake begin')
try:
    subprocess.run(['python3', 'waker.py'], timeout=2.5)
except subprocess.TimeoutExpired:
    pass
print('Awake end')

and I run python3 awake.py.

but get the following output:

Awake begin
The waker begins
start sleeper
Awake end
end sleeper

actually to be more accurate I get the first 3 lines printer immediatly, then 2.5 seconds later the 4th line prints and I get my bash prompt, then 7.5 seconds later end sleeper appears on my bash prompt.

how do I make it so killing a subprocess via timeout also kills the subprocesses run by that subprocess?

`run` should terminate the child process when the timeout expires. Does it terminate children as well? it doesn't seem so in your case. A workaround would be to use `Popen`, poll for the timeout, and kill process & children for instance like this: https://stackoverflow.com/a/27034438/6451573. This is very hacky... — Jean-François Fabre, Oct 24 '18 at 17:53

Jean-François Fabre · Accepted Answer · 2018-10-24T18:17:03.610

run should terminate the child process when the timeout expires. Does it terminate children as well? it doesn't seem so in your case. A workaround would be to use Popen, poll for the timeout, and kill process & children.

Seems that you cannot have it both ways: use run and be sure that all subprocesses are terminated (when you get a TimeoutException the process is already killed so you lose track of children)

proc = subprocess.Popen(args, stderr=errFile, stdout=outFile, universal_newlines=False)
wait_remaining_sec = 2.5

while proc.poll() is None and wait_remaining_sec > 0:
    time.sleep(0.5)
    wait_remaining_sec -= 0.5

if proc.poll() is None:
   # process is still there: terminate it and subprocesses:
   import psutil

   parent_pid = proc.pid  # we get the process pid
   parent = psutil.Process(parent_pid)
   for child in parent.children(recursive=True):
      child.kill()
   parent.kill()

The poll loop is better than a bare time.sleep(2.5) call because if process ends before timeout, you don't want to wait 2.5 seconds. This doesn't induce more than a 0.5s delay if the process ends before.

References:

is there a reason for `while proc.poll() is None and wait_remaining_sec > 0: time.sleep(0.5) wait_remaining_sec -= 0.5` and not just `time.sleep(2.5)` ? — Theo Walton, Oct 24 '18 at 18:05
yes, because if process ends before timeout, you don't want to wait 2.5 seconds. — Jean-François Fabre, Oct 24 '18 at 18:16

nested subprocess not stopping

1 Answers1