Python subprocess: how to retrieve the standard output

Question

So, I'm writing a python script in which I need to call another python script that prints several lines in the standard output. I want to store the output of the callee script in a list, and process them in the main script.

One simple way is to print the result into a file (file3) and read the file, like this

subprocess.call("./vecdiff.py file1 file2 > file3")

f = open("file3", "r")

How can I redirect the output directly into some lists in my main script?

possible duplicate of [Equivalent of Backticks in Python](http://stackoverflow.com/questions/1410976/equivalent-of-backticks-in-python) — John Kugelman, Mar 29 '13 at 15:58

score 3 · Answer 1 · answered Mar 29 '13 at 16:24

The simplest way is to call subprocess.check_output which calls the process and returns its output (or raises exception on non-zero return code, but you can still access the output through the exception object). It can easily deal with both STDOUT and STDERR (only STDOUT by default though).

Example from the manual:

>>> subprocess.check_output(["echo", "Hello World!"])
'Hello World!\n'

If your process is generating lots of data and you'd like to process it while it's running, you'll have to open the pipe and use communicate as described in other answers.

score 2 · Accepted Answer · answered Mar 29 '13 at 16:22

2

As below, redirect stdout and stderr of your child process to PIPE, then you can use 'communicate()' method to get the stdout and stderr of your child process from PIPE. The 'PIPE' is the pipe between father process and child process.

from subprocess import Popen, PIPE
p = Popen("./vecdiff.py file1 file2", stdout=PIPE, stderr=PIPE)
output, errput = p.communicate()

answered Mar 29 '13 at 16:22

Yarkee

9,086
5
28
29

Thanks,but I got an error: OSError: [Errno 2] No such file or directory: './vecdiff.py tr674.init.pdb TR674TS049_1.pdb ' – Vahid Mirjalili Mar 29 '13 at 18:16
You cannot pass the string with parameters embeded, as by default shell is disabled. Try to pass `["vecdiff.py", "file1", "file2"]` or, if you insist on the string, pass `shell=True` as an argument to `Popen`. The former is preferable. – tomasz Mar 29 '13 at 19:17

score 1 · Answer 3 · answered Mar 29 '13 at 16:03

1

If you use subprocess.Popen(), you can save the output with communicate(). For example:

proc = subprocess.Popen("./vecdiff.py file1 file2", stdout=subprocess.PIPE)
output = proc.communicate()

answered Mar 29 '13 at 16:03

Kevin D

431
4
7

2

You probably want to get rid of that `> file3` part, since you want the output of running `vecdiff.py` to go directly to stdout instead of a file. – Michael0x2a Mar 29 '13 at 16:04
I tried to use this method, but got an error: proc = subprocess.Popen("./vecdiff.py tr{0}.init.pdb TR{0}TS049_1.pdb ".format(tid), stdout=subprocess.PIPE) File "/usr/lib/python3.2/subprocess.py", line 745, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.2/subprocess.py", line 1361, in _execute_child raise child_exception_type(errno_num, err_msg) OSError: [Errno 2] No such file or directory: './vecdiff.py tr674.init.pdb TR674TS049_1.pdb ' – Vahid Mirjalili Mar 29 '13 at 18:22

score 0 · Answer 4 · answered Mar 29 '13 at 16:11

0

Easily done with plumbum:

from plumbum.cmd import python
cmd = python['./vecdiff.py']['file1', 'file2']
for line in cmd().splitlines():
    print line

answered Mar 29 '13 at 16:11

shx2

61,779
13
130
153

Python subprocess: how to retrieve the standard output

4 Answers4