Getting stdout to display called script containing input

Question

I was looking to implement a python script that called another script and captured its stdout. The called script will contain some input and output messages eg

print ("Line 1 of Text")
variable = raw_input("Input 1 :")
print "Line 2 of Text Input: ", vairable

The section of the code I'm running is

import subprocess

cmd='testfile.py'
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
so, se = p.communicate()
print(so)

The problem that is occurring is that the stdout is not printing until after the script has been executed. This leaves a blank prompt waiting for the user input. Is there a way to get stdout to print while the called script is still running?

Thanks,

From here: http://stackoverflow.com/questions/230751/how-to-flush-output-of-python-print import sys sys.stdout.flush() — Ilia Frenkel, Mar 30 '12 at 03:48
Which OS are you targeting? There are some weirdness on Windows with this type of application — Fuzz, Mar 31 '12 at 03:37

score 3 · Accepted Answer · edited May 23 '17 at 10:34

There are two problems here.

Firstly, python is buffering output to stdout and you need to prevent this. You could insert a call to sys.stdout.flush() in testfile.py as Ilia Frenkel has suggested, or you could use python -u to execute testfile.py with unbuffered I/O. (See the other stack overflow question that Ilia linked to.)
You need a way of asynchronously reading data from the sub-process and then, when it is ready for input, printing the data you've read so that the prompt for the user appears. For this, it would be very helpful to have an asynchronous version of the subprocess module.

I downloaded the asynchronous subprocess and re-wrote your script to use it, along with using python -u to get unbuffered I/O:

import async_subprocess as subprocess

cmd = ['python', '-u', 'testfile.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
so = p.asyncread()
print so,
(so, se) = p.communicate()
print so

When I run this script using python -u I get the following results:

$ python -u script.py
Line 1 of Text
Input 1:

and the script pauses, waiting for input. This is the desired result.

If I then type something (e.g. "Hullo") I get the following:

$ python -u script.py
Line 1 of Text
Input 1:Hullo
 Line 2 of Text Input:  Hullo

Thanks! It works fine, however if the contents of test file changes and has a second input then it stops working due to communicate() running until it reaches the end of file — Scavenger, Apr 02 '12 at 04:02

score 1 · Answer 2 · edited Mar 30 '12 at 14:12

You don't need to capture it's stdout really, just have the child program print out its stuff and quit, instead of feeding the output into your parent program and printing it there. If you need variable output, just use a function instead.

But anyways, that's not what you asked.

I actually got this from another stackoverflow question:

import subprocess, sys
cmd='testfile.py' 
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE) 
while True: 
    out = p.stdout.read(20) 
    if out == '' and p.poll() != None: 
        break 
    if out != '': 
        sys.stdout.write(out) 
        sys.stdout.flush()

First, it opens up your process: then it continually reads the output from p and prints it onto the screen using sys.stdout.write. The part that makes this all work is sys.stdout.flush(), which will continually "flush out" the output of the program.

@bear605 I run the code, but it doesn't work. The first print command doesn't appear and the script gets stuck with a keystroke not doing anything. It might be because I am running on windows. — Scavenger, Apr 01 '12 at 22:54

Getting stdout to display called script containing input

2 Answers2