python subprocess is overwriting file used for stdout - I need it to append to the file (windows)

Question

I want to append the STDOUT of subprocess.call() to an existing file. My code below overwrites the file -

log_file = open(log_file_path, 'r+')
cmd = r'echo "some info for the log file"'
subprocess.call(cmd, shell=True, stdout=log_file, stderr=STDOUT)
log_file.close()

I'm looking for the equivalent of >> in subprocess.call() or subprocess.Popen(). It's driving me crazy trying to find it..

UPDATE:

Following the answers so far I've updated my code to

import subprocess

log_file = open('test_log_file.log', 'a+')
cmd = r'echo "some info for the log file\n"'
subprocess.call(cmd, shell=True, stdout=log_file, stderr=subprocess.STDOUT)
log_file.close()

I'm running this code from the command line in windows -

C:\users\aidan>test_subprocess.py

This adds the text to the log file. When I run the script again, nothing new is added. It still seems to be overwriting the file..

This is a deeper issue on Windows and append mode, see https://bugs.python.org/issue45237 for workarounds. — Wolfgang Kuehn, Sep 19 '21 at 18:33

Martijn Pieters · Accepted Answer · 2012-05-23T16:37:21.693

7

Use the 'a' append mode instead:

log_file = open(log_file_path, 'a+')

If you still see previous content overwritten, perhaps Windows needs you to explicitly seek to the end of the file; open as 'r+' or 'w' and seek to the end of the file:

import os

log_file = open(log_file_path, 'r+')
log_file.seek(0, os.SEEK_END)

edited May 23 '12 at 16:37

answered May 23 '12 at 15:09

Martijn Pieters

1,048,767
296
4,058
3,343

Thanks for the input - that makes sense, but somehow it still fails (I've updated my question with a complete script). – Aidan Ewen May 23 '12 at 16:16
I've updated my answer to include a seek-to-end alternative to opening the file for appending. – Martijn Pieters May 23 '12 at 16:37

score 4 · Answer 2 · edited May 23 '12 at 19:39

4

Modify how you open log_file_path. You are opening the file for reading and writing 'r+'. Use the 'a' append mode instead of 'r+':

log_file = open(log_file_path, 'a+')

edited May 23 '12 at 19:39

sblom

26,911
4
71
95

answered May 23 '12 at 15:09

NPE

486,780
108
951
1,012

python subprocess is overwriting file used for stdout - I need it to append to the file (windows)

2 Answers2

Linked