I'm having a similar problem described here: subprocess stdin buffer not flushing on newline with bufsize=1
My script is not outputting line by line when I specify bufsize=1. It's infact outputting when the buffer fills. I want to read line by line in realtime.
I've tried the below on Linux Mint and CentOS 7. test.py is what I run, it calls the 'script' file which is executable.
test.py
#!/usr/bin/python
import subprocess
import sys
process = subprocess.Popen('/root/script',bufsize=1, stdout=subprocess.PIPE, stderr=subprocess.PIPE,close_fds=True, universal_newlines=True)
for line in iter(process.stdout.readline, b''):
print line
script
#!/usr/bin/python
import time
for i in range(0,999999):
print str('This is a short line, ' + str(i) + '\n')
time.sleep(0.01)
This outputs large numbers of lines at once with pauses between the output. The first large chunk ends at line 154
However, If I make the output line much larger in the script file:
print str('This is a long line................................................................................................. ' + str(i) + '\n')
The first large chunk ends at line 66
Why is my buffer size being ignored?
