Python subprocess.check_call - how to direct stdout and stderr to _both_ screen and log file?

Question

I use Python 2.7. This is what I'm doing currently to launch other programs from Python, redirecting STDOUT and STERR to LOGFILE:

    try:
    # Execute command and print content to LOGFILE
        tempname = os.path.abspath('./text.txt')
        TEMPFILE = open(tempname, 'wb')
        print 'Executing: ', command
        subprocess.check_call(command, shell = True, stdout = TEMPFILE, stderr = TEMPFILE)
        TEMPFILE.close()
        LOGFILE.write(open(tempname, 'rU').read())
        LOGFILE.close()
        os.remove(tempname)
    except Exception as errmsg:
    # If fails then print errors to LOGFILE
        TEMPFILE.close()
        LOGFILE.write(open(tempname, 'rU').read())
        os.remove(tempname)
        print messages.crit_error_bad_command % command, '\n', str(errmsg)
        print >> LOGFILE, messages.crit_error_bad_command % command, '\n', str(errmsg)
        LOGFILE.close()

First of all, I wasn't sure my above script was the best solution. I had to use a temp file because I want to capture the log even in the case the subprocess.check_call call fails. If you have ideas how to improve the above script I'd appreciate it.

Moreover, I would like to change this so that STDOUT and STDERR should go to the screen as normal AND also to the log file. How do I do this? Note that if I don't specify STDOUT, I'll see things like "There is an error, do you wish to continue [y/n]?" on the screen and then I can react to it. Right now since everything goes to the log, I do not see anything on the screen. The answer to my question here should help resolve this case.

Thanks.

What about using the [logging](http://docs.python.org/library/logging.html) module? — brandizzi, Jun 27 '12 at 22:06
You can use `subprocess Popen.communicate()`, which returns `stdout,stderr`. That way you can do whatever you want to it (log it, print it) — TJD, Jun 27 '12 at 22:22
To further add to @TJD, check_call is a convenience method. Its too limited for this case, which is why you should be using a Popen object — jdi, Jun 27 '12 at 22:34
Can you guys elaborate with code samples? I'm not familiar with Popen. — cinny, Jun 27 '12 at 22:43
With Popen, would subprocess waits until the process is fully completed before it moves on to the next Python command? Please write a full answer if possible. Thanks. — cinny, Jun 27 '12 at 22:45
Do you want to answer manually in real time to questions from a subprocess or you can provide answers beforehand? — jfs, Jun 28 '12 at 02:08

score 1 · Answer 1 · edited May 23 '17 at 12:20

1

I'll see things like "There is an error, do you wish to continue [y/n]?" on the screen and then I can react to it.

To redirect subprocess' stdout and to interact with the process either manually or using predefined answers you could use pexpect:

from contextlib import closing
import pexpect

with open('text.txt', 'rb') as logfile:
    with closing(pexpect.spawn(command, logfile=logfile)) as child:
        # call here child.expect(), child.sendline(), child.interact(), etc

Implementing the same functionality using subprocess.Popen is not trivial.

edited May 23 '17 at 12:20

Community

1
1

answered Jun 28 '12 at 03:18

jfs

399,953
195
994
1,670

Actually I usually do not expect to have to interact, hence my original approach. BUT, sometimes it happens out of my control, and without being able to see the question, the program appears to be hanging instead, which is not good. – cinny Jun 28 '12 at 12:47

Python subprocess.check_call - how to direct stdout and stderr to _both_ screen and log file?

1 Answers1