Why does subprocess hang when reading from stdout

Question

I have a script called 'my_script.py' with the following contents:

my_input = ''
while my_input != 'quit':
    my_input = raw_input()
    print(my_input)

Then in the console, the following commands:

from subprocess import *
p1 = Popen(['python', 'my_script.py'], stdin=PIPE)
p1.stdin.write('some words\n')

prints "some words", but if instead I write

from subprocess import *
p2 = Popen(['python', 'my_script.py'], stdin=PIPE, stdout=PIPE)
p2.stdin.write('some words\n')
p2.stdout.readline()

the shell will hang and I have to terminate it manually. How can I get this to work if I want to be able to access the stdout of the script? I'm using python2.7

Edit: To clarify my question, the above snippet will run properly for other executables that have an I/O loop (the particular one I'm working with is the stockfish chess engine). Is there a way I can modify my_script.py such that the above snippet will run properly? Using

Popen(['python3', 'my_script.py'], ...)

will work, but is it not possible using Python 2.7?

Samuel · Answer 1 · 2016-05-16T04:41:29.123

0

It could happen due to deadlock in readline.

You need to use communicate method to read asynchronously.

Example:

 def run_shell_command(cmd, params):
        cmdline = [cmd]
        p = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        stdout, stderr = p.communicate()
        if p.returncode != 0:
            raise RuntimeError("%r failed, status code %s stdout %r stderr %r" % (
                cmd, p.returncode, stdout, stderr))
        return stdout.strip()  # This is the stdout from the shell command

To execute command in background you can use following example:

def run_shell_remote_command_background(cmd):
    cmdline = [cmd]
    cmdline = cmdline + params.split(' ')
    subprocess.Popen(['cmdline'])

edited May 16 '16 at 04:41

answered May 16 '16 at 04:15

Samuel

3,631
5
37
71

1

unfortunately `communicate` only allows one round of input to the subprocess, where I need multiple. – Adam Venis May 16 '16 at 04:27
@AdamVenis do you mean multiline output? – Samuel May 16 '16 at 04:31
no, in your example if i wanted send input with `p.communicate('foo')`, I wouldn't then be able to send `p.communicate('bar')` afterwards without opening a new subprocess. This is what I understand to be the drawback of using `communicate`, because it waits for the subprocess to finish executing entirely. – Adam Venis May 16 '16 at 04:35
@AdamVenis Got it, you want to execute subprocess in the background. Then `Popen(['python3', 'my_script.py'])` will be good from 2.7 as well – Samuel May 16 '16 at 04:38

Why does subprocess hang when reading from stdout

1 Answers1