Communication with process using pipes in Python

Question

I have a process with which I can communicate on the command line like this:

% process -
input
^D^D
output

So: I start the process, type some input and after hitting Ctrl-D twice, I get the output. I want to make a Python wrapper around this process. I created this:

from subprocess import Popen, PIPE

p = Popen('process -', stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=True)
while True:
        input = raw_input('Enter input: ')
        p.stdin.write(input)
        p.stdin.close()
        p.wait()
        output = p.stdout.read()
        print output

This works the first time, but after that I get:

Traceback (most recent call last):
  File "test.py", line 7, in <module>
    p.stdin.write(input)
ValueError: I/O operation on closed file

Is there another way to interact with this process without closing the file?

The problem is a bit more complicated. As I see it now you have to remove those closing lines, add `p.stdin.flush()` and catch the SIGINT signal. And when it occures you have to pass it to the subprocess and **then** `p.wait()` (or better `p.communicate()`), print the output and close the process. — freakish, Feb 02 '15 at 13:24

score 0 · Answer 1 · answered Feb 02 '15 at 12:55

0

p.wait() will wait until the subprocess has exited prior to returning, so on the second iteration in your script, p has exited already (and has therefore closed p.stdin).

answered Feb 02 '15 at 12:55

Thomas Orozco

53,284
11
113
116

score 0 · Answer 2 · answered Feb 02 '15 at 14:56

If the proccess you're wrapping ends after the first ouput, the comunication will fail to the second. Due all pipes (stdin and stdout) will be closed. Hence the error:

ValueError: I/O operation on closed file.

Each time you try to send input to the wrapped process, this must be expecting that input and pipes must be opened.

On the other hand is what Thomas said in his answer, p.wait() is not the way to go for repetitive input/output strategy.

You can't use subprocess.Popen.communicate() neither, due it calls subprocess.Popen.wait() internally.

You can try use p.stdin.write and p.stdout.read here you have a good article about the subject: Writing to a python subprocess pipe

score 0 · Answer 3 · edited May 23 '17 at 12:12

To emulate the shell session:

$ process -
input
^D^D
output

In Python, using check_output():

#!/usr/bin/env python3
from subprocess import check_output

out = check_output(['process', '-'], input='input\n', universal_newlines=True)
print(out, end='')

Ctrl+D is recognized as EOF (terminate the input) by a Unix terminal ($ stty -a -- look for eof = ^D and icanon in the output). If you need to type Ctrl+D twice (at the beginning of a line); it might indicate a bug in the process program such as "for line in sys.stdin: doesn't notice EOF the first time" Python bug.

Communication with process using pipes in Python

3 Answers3