0

I am trying to use subprocess to start a command on the command-line for which I want to have the output written into some file. So, in other words, I want to have e.g. the following command executed within python:

python my_code.py --arg1 > output.txt

I tried the following:

import subprocess
cwd = "/home/path/to/the/executable"
cmd = "python my_code.py --arg1"
with open('output.txt', "w") as outfile:
    subprocess.Popen(cmd.split(), stdout=outfile, cwd = cwd)

but the output file stayed empty. How to else do it (not blocking!)?

Addition:

My guess is that the output file is created, but is being closed as soon as the above code finishes. Therefore, no output to that file...

Alex
  • 41,580
  • 88
  • 260
  • 469
  • what's `my_code.py`? – Daniel Jun 14 '16 at 18:33
  • Its a python script (a service) which I want to start from another piece of code. Its just some code, does not matter... – Alex Jun 14 '16 at 18:36
  • `subprocess.Popen(cmd.split(), stdout=outfile, cwd = cwd).wait()` – Nizam Mohamed Jun 14 '16 at 18:45
  • I am looking for a NON-BLOCKING way to do this. I do not want to wait until the call of `my_code.py` has finished. I want the above code snippet to end as soon as it has finished... – Alex Jun 14 '16 at 18:46
  • I found something interesting while searching about it. http://stackoverflow.com/questions/17329648/unable-to-write-to-stdin-in-subprocess – Rahul K P Jun 14 '16 at 18:52
  • @SvenMarnach - closing the file in the parent doesn't affect the file in the child. – tdelaney Jun 14 '16 at 18:53
  • This should work, at least on linux. The subprocess will continue execution and write to the file. Does your parent script exit immediately? Do you see the program using, say, `ps x`? Could it have failed? – tdelaney Jun 14 '16 at 18:56
  • `cmd.split()` is sensitive to things like spaces in command parameters. `shlex.split(cmd)` may be a better option. – tdelaney Jun 14 '16 at 18:59
  • If the main process ends before child, the child keeps running. The child should flush its stdout as soon as it writes to see the output in the file – Nizam Mohamed Jun 14 '16 at 19:00
  • @NizamMohamed is correct. The child writes to `stdout` but it only appears in the output file when a full (system dependent) block is written or when the child exits. – tdelaney Jun 14 '16 at 19:02

1 Answers1

2

sys.stdout is buffered by default. You can disable it by passing -u to python. 

import subprocess
cwd = "/home/path/to/the/executable"
cmd = "python -u my_code.py --arg1"
with open('output.txt', "w") as outfile:
    subprocess.Popen(cmd.split(), stdout=outfile, cwd = cwd)
Nizam Mohamed
  • 8,751
  • 24
  • 32
  • This is a great answer but is only needed if you want "real time" updates of the file. The system purposely delays file writes as it is more efficient. – tdelaney Jun 14 '16 at 19:18