0

From Python, I want to execute a bash command (maybe with subprocess.Popen()) and I want to print the output and also to save the output in a variable or in a file, at the same time. This command returns a lot of text and it takes some hours to run.

If I redirect the output to a file, can I immediately print every new line which appears in this file?

Ciprian Vintea
  • 448
  • 2
  • 4
  • 18
  • 1
    See the `tee` command. – isedev Oct 04 '14 at 09:39
  • related: [How do I push a subprocess.call() output to terminal and file?](http://stackoverflow.com/q/25963074/4279) – jfs Oct 04 '14 at 10:17
  • related: [Displaying subprocess output to stdout and redirecting it](http://stackoverflow.com/q/25750468/4279) – jfs Oct 04 '14 at 10:18
  • related: [Python subprocess get children's output to file and terminal?](http://stackoverflow.com/q/4984428/4279) – jfs Oct 04 '14 at 10:19
  • related: [Subprocess.Popen: cloning stdout and stderr both to terminal and variables](http://stackoverflow.com/q/17190221/4279) – jfs Oct 04 '14 at 10:20
  • @isedev: `tee` might not help: 1. the child process may change what it prints if it detects that the output is a pipe (`| tee`) 2. the output may be not immediate due to block buffering 3. the child process may print to terminal directly (outside of stdout/stderr). 4. You may want to save stdout/stderr separately. See the references above that show how to resolve these issues – jfs Oct 04 '14 at 10:23

1 Answers1

0

If you want to do this all in Python, without the tee command, you can. Basically you need to set stdout=PIPE (see the subprocess docs), then use select() on it, to be notified whenever there is a chunk of data to read. When there is data available, read it and write it to a file and print it at the same time. A solution to a similar problem which you can adapt for this purpose is here: https://stackoverflow.com/a/12272262/4323

Community
  • 1
  • 1
John Zwinck
  • 239,568
  • 38
  • 324
  • 436