Calling external program through Python and passing the output to stdout

Question

How can I call an external program which is written in bash script in such a way that the output produced by that script is available in sys.stdout so that I can log the output in a file through python.

For example. I now call them through the following snippet

if os.name == 'nt':
    path = module_dir_run+'/run.bat'
else:
    path = module_dir_run+'/run.sh'
if os.path.isfile(path):
    if (splitargs.arg):
        try:
            call([path, splitargs.arg])
        except:
            pass
    else:
        try:
            call([path])
        except:
            pass
else:
    print "Not found : " + path

when I store the value of sys.stdout = file(filename, "w") it stores whatever which the python outputs, but not what the script outputs.

NOTE: The script which i am trying to run is an interactive script, so after the call has ended, and the control has come back to python, how can i get all what is written in the terminal? Any suggestions?

Is it mandatory to write the output to `stdout`? I would suggest that just write it to a file and then read the file. — Timothy, Jan 10 '13 at 09:51
@MartijnPieters yes, but maybe its a function which implements it with some sort of argument or maybe its something else... — Inbar Rose, Jan 10 '13 at 09:53
it is required, since if someone calls the function like run modulename > output.txt — Subho Halder, Jan 10 '13 at 09:56
@MartijnPieters the script has interactive features, where user has to interact, will those be recorded as well? — Subho Halder, Jan 10 '13 at 10:05
@SunnyRockzzs: You cannot redirect stdout output *and* interact with a human being; they cannot see the output to interact with the script. — Martijn Pieters, Jan 10 '13 at 10:07
@MartijnPieters so there is no way by which when the script is finished running, and the control comes back to python, it records all which is present in between in the terminal ? — Subho Halder, Jan 10 '13 at 10:11
@SunnyRockzzs: Not and support output redirection, no. You *can* use `PIPE` on `stdout`, print that and write it to a file separately. If you run `python script.py > output.txt` the end-user won't get to see the output interactively. — Martijn Pieters, Jan 10 '13 at 10:13
@MartijnPieters I am not doing that, I am using the cmd module for a custom commandline based program, so I start the program with `python run.py` and then in the interface I type `run module_name` which runs the specified function and calls the script ! now I am trying to add a feature like `run module_name > output.log` — Subho Halder, Jan 10 '13 at 10:21
You need to explain your scenario better. I have a feeling that whatever it is you're trying to solve with this elaborate script can be solved with a much simpler implementation. So, please include more details and we'll be able to help better with alternate solutions. — Zoran Pavlovic, Jan 10 '13 at 13:25

score 2 · Accepted Answer · answered Jan 10 '13 at 10:02

2

I always use subprocess.Popen() to run another program from inside a Python script. Example:

import subprocess

print "Starting process..."
process = subprocess.Popen(["echo", "a"], shell=False)
process.wait()
print "Done"

You can redirect the output of process to another file like this:

import subprocess

print "Starting process..."
with open("./out.log", "w") as f:
    process = subprocess.Popen(["echo", "a"], shell=False, stdout=f)
    process.wait()
print "Done"

Redirecting stderr is also possible through 'stderr' parameter.

When you have the sys.stdout in current script redirected to write to your own file, you can do this to redirect it in your subprocess too:

import subprocess
import sys

sys.stdout = open("./out.log", "w")
print "Starting process..."
sys.stdout.flush()
process = subprocess.Popen(["echo", "a"], shell=False, stdout=sys.stdout)
process.wait()
print "Done"

answered Jan 10 '13 at 10:02

ZalewaPL

1,104
1
6
14

the script has interactive features, where user has to interact, will those be recorded as well? If i pass the stdout to a file descriptor befor hand, user cannot interact with the script ! – Subho Halder Jan 10 '13 at 10:07
So you want the output of your subprocess to be visible both on the console and written to the log file? As for the interaction, current stdin is passed implicitly and user will be able to type in commands from keyboard. – ZalewaPL Jan 10 '13 at 10:16
Yes, that's what i want ! but i want the user to interact natively with the script, and when the script has ended, a variable or stdout can store whatever it is writen in the terminal. – Subho Halder Jan 10 '13 at 10:18
It can be done but it can be a bit of a trouble and you'll have to rely on your subprocess to flush stdout immediatelly when something new appears. Your Python script needs to be able to print to the console, so if you redirected the sys.stdout you will need to keep the original one in a variable somewhere so you can write subprocess's output to it. Then you would need to redirect the subprocess' stdout to a separate file, and in the main process keep reading that file in a loop until process exits. That way you can do anything you want with the output. – ZalewaPL Jan 10 '13 at 10:31
i didn't get you, could you provide a simple example? – Subho Halder Jan 10 '13 at 10:39

score 1 · Answer 2 · answered Jan 10 '13 at 10:00

1

You can write the outputs of such a call into a variable using e.g.:

variable = subprocess.Popen(
    ['ls', 'mydir/'],  # note that the command is split on whitespace
    stdout=subprocess.PIPE).communicate()[0]

answered Jan 10 '13 at 10:00

dan_kaufhold

637
5
16

the script which i am trying to run has interactive features, where user has to type in their name, etc, will that be recorded too ? – Subho Halder Jan 10 '13 at 10:08
@SunnyRockzzs so you want to make it callable like ``script.py arg1 arg2``? If I understand correctly, maybe this [stack overflow question](http://stackoverflow.com/questions/4429966/how-to-make-a-python-script-pipeable-in-bash) will take you further – dan_kaufhold Jan 10 '13 at 11:33
I think what he means is that the script he's trying to run reads in from stdin. Note his use of the word "interactive". So one needs to redirect stdin back to the shell running the python script. – Zoran Pavlovic Jan 10 '13 at 13:10

score 0 · Answer 3 · answered Jan 10 '13 at 09:59

As Martijn Pieters already states, you'll have to provide an alternative stdout for your call() call.

There are essentially 2 ways: * either provide an already opened file (as you have changed your sys.stdout, this will probably be fine) * use stdout=subprocess.PIPE and read and treat the stdout yourself. (Not via subprocess.call() then, but via the object returned by subprocess.Popen().)

Note that changing sys.stdout won't really change your process's stdout from the view of the OS, but instead will order your program to output everything on a non-usual file descriptor.

score 0 · Answer 4 · edited May 23 '17 at 11:44

0

This might be of use. Particularly because most of the other answers don't understand that you need to redirect your parent shell's stdin to the child process's stdin because the programs/scripts you're calling are "interactive".

Python and subprocess input piping

edited May 23 '17 at 11:44

Community

1
1

answered Jan 10 '13 at 13:23

Zoran Pavlovic

1,166
2
23
38

Calling external program through Python and passing the output to stdout

4 Answers4