3

A program continuously prints to standard output one line at a time.

I am trying to read and process one line at a time of this input without having to wait for the program to complete.

As an example, the below writeOutput.py writes one line at a time to stdout (waiting between each line between 1 and 3 seconds).

Calling ./writeOutput.py | ./processEachLine.py requires writeOutput.py to complete before processEachLine.py is able to start processing the first line.

Is there anyway to achieve this in python? Even by calling writeOutput.py directely within the python program instead of using a pipe?

Any help would be highly appreciated.

writeOutput.py

#!/usr/bin/env python
import random
import time

i = 0
while i < 5:  
    n = int(1 + (random.random()*10) % 3)
    i += 1
    time.sleep(n)
    print(str(n) + " test")

processEachLine.py

#!/usr/bin/env python    
import sys

while 1:
    line = sys.stdin.readline()
    if not line:
      break
    print(">>" + line)
Jason Sundram
  • 12,225
  • 19
  • 71
  • 86
tartifletteblvd
  • 51
  • 4

1 Answers1

5

Instead of 

#!/usr/bin/env python

use

#!/usr/bin/env python -u
uselpa
  • 18,732
  • 2
  • 34
  • 52
  • Thank you. #!/usr/bin/env python –u would create an error: "/usr/bin/env: python -u: No such file or directory" An easy workaround is to use python -u writeOutput.py | python -u processEachLine.py Or add #!/usr/bin/python -u Many thanks for the help. – tartifletteblvd Jun 02 '13 at 18:10
  • That's strange, it works well for me (tried on OS X and Linux). What OS are you on? – uselpa Jun 02 '13 at 18:37
  • 1
    This is probably necro, but the dash is probably bad - managed to recreate the error by copying the dash from tartifletteblvd's comment – Nitz Jan 27 '15 at 21:51
  • @Nitz You're right, he does seem to use a character that looks like a dash but isn't. – uselpa Jan 28 '15 at 11:50
  • Actually, the hashbang mechanism parses the rest of the first line as one program name (`/usr/bin/env`) and one argument (`python -u`, space included, which is not a program as you can understand). So `#!/usr/bin/python -u` works, while `#!/usr/bin/env python -u` doesn't. See also: http://stackoverflow.com/a/4304187/6899 – tzot Mar 23 '16 at 15:24
  • @tzot It's been a long time but I remember I tried my answer before posting it (I always do) and it *did* work. I also assume it worked for the OP since he accepted the answer. Did you actually try this yourself, and if so, on what platform (I use OS X)? – uselpa Mar 23 '16 at 16:50
  • In the first comment of your answer, the OP stated that it didn't work for them, like it doesn't for me (on linux). Anyway, I should be more clear that the success depends on the system, that is why I included a link to a more descriptive answer. – tzot Mar 25 '16 at 02:19