Python - Unable to pass an opened file to Popen as stdout

Question

I'm currently trying to redirect the standard output of the subprocess.Popen object to an opened file, so I followed the instructions found in many websites such as this.

However, the Popen.stdout is not correctly assigned for some reason

My code went as follows:

def foo(file):
    print(file)              # printA

    proc = subprocess.Popen('command, stdout=file) # i've tried with both shell=True/False
    print(proc)              # printB
    print(proc.stdout)       # printC
    return proc

def main():
    file = open('path', 'w')
    p = foo(file)
    print(p.stdout)          # printD

The result is as of follows

printA: <_io.TextIOWrapper name='path' mode='w' encoding='UTF-8'>
printB: <subprocess.Popen object at 0x161966d0>
printC: None
printD: None

from what I read in the python doc here, the default for stdout is None if no PIPE is assigned to it. But since this here shows that apparently we can pass an opened file as the argument to stdout, I don't understand why my code doesn't work.

Please help.

the printA clearly shows that what open returns is *NOT* a standard file (with an OS filedescriptor, which is what subprocess really needs). For this to work, you have to pass a real file. — deets, May 21 '14 at 19:56
W: 1,8:foo: Redefining built-in 'file' W: 10,4:main: Unused variable 'f' — jhermann, May 21 '14 at 20:05
Forget my answer - I wasn't aware of the io-modul's meaning here, it looked like an in-memory-wrapper like StringIO, which doesn't work. Probably @zmo's answer is good. — deets, May 21 '14 at 20:06

score 2 · Answer 1 · answered May 21 '14 at 20:03

2

Your code has a few flaws, corrected below:

def main():
    with open('path', 'w') as f:
        subprocess.Popen(['command'], stdout=f)

You're giving the file object as a parameter to your function, instead of the f object. Then in your function you're shadowing the global file object that you give to Popen, though incidently, it's the real file object.

answered May 21 '14 at 20:03

zmo

24,463
4
54
90

Though in py3 there's no more global `file` object. But your SSCCE is still wrong. – zmo May 21 '14 at 20:16
I think it was a typo. You would have a different printout in PrintA – Oleg Gryb May 21 '14 at 20:30
THANKS!!!, this actually worked. However, I do have an additional question: I have tried to assign the popen object to x, and killed the process just before my program exit. As I do that, the file no longer gets overwritten. It seem that the file is not written along with the input, but rather stored in the buffer and is then written into the file at the end. Any idea what happened? – user1948847 May 21 '14 at 20:40
It is what you say, the context manager flushes all the data into the file upon exiting the block. – zmo May 21 '14 at 20:50
@zmo is there any way to let the manager flush the data whenever it captures it? And also, sorry that I have to change the accepted answer to Oleg's anwser, eryksun has a point. – user1948847 May 21 '14 at 20:56
Sadly not, if you want that you'll need to handle reading stdout and writing to the file yourself – zmo May 21 '14 at 20:58
You can ask that as another question with the code and your problem – zmo May 21 '14 at 21:00
Your answer doesn't make sense with [the current revision of the question (`f` -> `file`)](http://stackoverflow.com/revisions/23792576/3). – jfs May 22 '14 at 00:24

Oleg Gryb · Accepted Answer · 2014-05-21T21:26:05.607

2

I've run a simpler example and this is what I found:

>>> p = subprocess.Popen('ls', stdout=open('moo','w'))
>>> str(p.stdout)
'None'

However, file 'moo' does have the content of my current directory. After a useful hint from the user @eryksun (see comments below) and reading the source code, I've realized that p.stdout variable will be assigned only if PIPE is passed in the stdout parameter. In all other cases it will be set to None and this is exactly what the example above has demonstrated. The bottom line - everything works as expected.

Also, please notice that if what the previous answer has suggested is correct you would get

<type 'file'>

in printA, so I don't think that the previous answer is correct.

edited May 21 '14 at 21:26

answered May 21 '14 at 20:41

Oleg Gryb

5,122
1
28
40

@eryksun - you're right. self.stdout in Popen will be set only if you pass PIPE as a parameter, but in essence it doesn't change much because what the author was asking about works as he wanted, so it was a "false alarm" so to speek. I'll update the answer. – Oleg Gryb May 21 '14 at 21:15
@eryksun - I don't care about points. There are many non-adequate people around, who are downvoting without even explaining a reason :) Thanks for the code link - I've enjoyed reading it. – Oleg Gryb May 21 '14 at 21:24
You should change the open statement to the context manager to avoid any kind of side effects on closing the file… – zmo May 21 '14 at 22:13
Thanks @zmo. The purpose of the short example above was to demonstrate that redirection to stdout works as expected, not to provide a perfect code example. – Oleg Gryb May 21 '14 at 22:57
[the documentation for `Popen.stdout` explicitly covers it](https://docs.python.org/3.4/library/subprocess.html#subprocess.Popen.stdout); you do not need to read the source code to find out that `p.stdout` is None in this case. Please, use `with`-statement for the file. – jfs May 22 '14 at 00:15

Python - Unable to pass an opened file to Popen as stdout

2 Answers2