1

I want to iterate over stdout of a subprocess while the subprocess is still running. My current approach looks like this:

proc = subprocess.Popen(['long_running_command'], stdout=subprocess.PIPE)
for line in iter(proc.stdout.readline, ''):
    # do some stuff
    print(line.rstrip())

At the moment the code in the for-loop will only be executed when the subprocess is finished. Is it possible to iterate over stdout of a runnnig subprocess and if yes how is it done? I know there are many existing threads with similar questions. However none of the provided answers seem to work for me.

Mark Amery
  • 143,130
  • 81
  • 406
  • 459
f1nan
  • 156
  • 10
  • 1
    This looks similar to [this](https://stackoverflow.com/a/6414278/1832539) answer. Did you see that yet? – idjaw Jul 25 '17 at 15:29
  • 1
    How much data does the subprocess produce? Normal output buffering would send every line at best, not every character. Buffering [can be prevented](https://stackoverflow.com/a/40453613/223424). – 9000 Jul 25 '17 at 15:30
  • take a peak at https://stackoverflow.com/questions/4417546/constantly-print-subprocess-output-while-process-is-running – Jacobr365 Jul 25 '17 at 15:31
  • @idjaw: thats pretty much what I did. Nevertheless i tried the answer the exact same way as it was suggested. I did not work. – f1nan Jul 25 '17 at 15:46
  • @9000: It produces a few lines of text. The solution you suggested worked. Thanks. However is there a way that does this without relying on external commands? – f1nan Jul 25 '17 at 15:52
  • @Jacobr365: I saw that already, but thanks. – f1nan Jul 25 '17 at 15:56
  • @RobertMoskal: I know there are a lot of similar threads. However, I did not find a satisfying answer. – f1nan Jul 25 '17 at 15:57
  • @f1nan: I suppose it's the OS mechanics, not Python's. You can consider [opening an explicit pipe](https://docs.python.org/2/library/os.html#os.mkfifo), set it unbuffered, and pass it as `stdout` to the process. This won't depend on an external program. – 9000 Jul 25 '17 at 16:12
  • @9000: Thank you, I'll try that. – f1nan Jul 25 '17 at 16:19
  • @9000: That doesn’t do anything: only the file descriptor is preserved across `exec`, and it doesn’t do the buffering anyway. – Davis Herring Sep 02 '19 at 03:54

0 Answers0