Using subprocess.Popen to start a program in Windows - subprocess sometimes gets stuck - is my script stable?

Question

As part of my Python (3.8.2) script, I start a program in Windows (10) via command line. This program runs a simulation, exports data, and closes. My script does this hundreds of times. Very occasionally, the cmd window appears but displays nothing (the Windows program does not seem to be doing anything). I am trying to determine whether the problem is with the external program, or my script.

I have searched for other answers but the subprocess world is proving difficult to understand. I use Python for some basic moving/interpreting data. I saw this answer (Understanding Popen.communicate), which includes

Note: it is a very fragile code if read/write are not in sync; it deadlocks.

This seems relevant but I am not in a position to understand & implement it.

My function is:

def run_program(args):
    cmd = #defined by args
    p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
    stdout, stderr = p.communicate()
    variables = #read from some datafiles created by the program that was run
    return variables

Is there any clear reason why the subprocess gets stuck? (I don't know if the correct term is blocks or hangs).

score 0 · Answer 1 · answered Mar 18 '21 at 10:27

A working solution, with inspiration from How to kill subprocess if no activity in stdout/stderr

def run_program(args, timeout):
    cmd = [] #defined by args
    if timeout == 'default':
        timeout = 600
    tries = 0
    while tries < 4:
        tries += 1
        try:
            p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
            t = 0
            while p.poll() != 0:
                if t > timeout:
                    raise RuntimeError('Timeout')
                    break
                time.sleep(1)
                t+=1
            successful = True
        except RuntimeError:
            kill = subprocess.Popen("TASKKILL /F /PID {pid} /T".format(pid=p.pid))
            # some actions for this condition
            successful = False
        if successful == True:
            # some actions for this condition
            break
    return variables, t

I use the timeout variable to pass information about the last run time in/out of the function.

Using subprocess.Popen to start a program in Windows - subprocess sometimes gets stuck - is my script stable?

1 Answers1