31

I can successfully redirect my output to a file, however this appears to overwrite the file's existing data:

import subprocess
outfile = open('test','w') #same with "w" or "a" as opening mode
outfile.write('Hello')
subprocess.Popen('ls',stdout=outfile)

will remove the 'Hello' line from the file.

I guess a workaround is to store the output elsewhere as a string or something (it won't be too long), and append this manually with outfile.write(thestring) - but I was wondering if I am missing something within the module that facilitates this.

Jdog
  • 10,071
  • 4
  • 25
  • 42

4 Answers4

36

You sure can append the output of subprocess.Popen to a file, and I make a daily use of it. Here's how I do it:

log = open('some file.txt', 'a')  # so that data written to it will be appended
c = subprocess.Popen(['dir', '/p'], stdout=log, stderr=log, shell=True)

(of course, this is a dummy example, I'm not using subprocess to list files...)

By the way, other objects behaving like file (with write() method in particular) could replace this log item, so you can buffer the output, and do whatever you want with it (write to file, display, etc) [but this seems not so easy, see my comment below].

Note: what may be misleading, is the fact that subprocess, for some reason I don't understand, will write before what you want to write. So, here's the way to use this:

log = open('some file.txt', 'a')
log.write('some text, as header of the file\n')
log.flush()  # <-- here's something not to forget!
c = subprocess.Popen(['dir', '/p'], stdout=log, stderr=log, shell=True)

So the hint is: do not forget to flush the output!

Joël
  • 2,723
  • 18
  • 36
  • Note on the above post: after some trials, it seems that `subprocess.Popen` needs something really like a file for its `stdout` and `stderr`arguments, so giving to them a custom object with only a `write` method is not sufficient. I tried, but a `fileno` function is required, and I'm not good enough to emulate this on a class of my own. – Joël Oct 13 '11 at 06:54
  • 5
    I have been attempting to use this method but I have discovered that for some reason, each time I run the external process with a file which I have definitely opened for appending, the output of the process writes from the beginning of the file and overwrites any information there. So, in order to use this solution, one must call log.seek( 0, os.SEEK_END ) immediately after opening the file. – Luke Oct 31 '12 at 15:13
2

Well the problem is if you want the header to be header, then you need to flush before the rest of the output is written to file :D

user3849635
  • 21
  • 1
0

Are data in file really overwritten? On my Linux host I have the following behavior: 1) your code execution in the separate directory gets:

$ cat test
test
test.py
test.py~
Hello

2) if I add outfile.flush() after outfile.write('Hello'), results is slightly different:

$ cat test
Hello
test
test.py
test.py~

But output file has Hello in both cases. Without explicit flush() call stdout buffer will be flushed when python process is terminated. Where is the problem?

Aliaksei Ramanau
  • 909
  • 1
  • 9
  • 16
0

I faced the same problem and resolved it with the code below

      with open("outfile.log", 'w') as f: # use 'a' if you need to append
            subprocess.Popen(['ls', '-la'], shell=False,
                             stdout=f.fileno(),
                             stderr=f.fileno())

Greetings

0xFF
  • 21
  • 3