I would like to know if there is any way to handle the case of non blocking as well as streaming the stdout and stderr from a python subprocess. I have gone through couple of articles over this website, but none has an answer to this scenario. It either shows by running the read in another thread, so that the main thread is left unaffected (or) just how to stream the data from the process. My issue is a combination of both. I guess the problem is at the place of read() or readline() as they are blocking one way or the other during the output streaming.
I want to have a streamable object that's my main concern. I am attaching the references I have been through for further reference.
script_process = subprocess.Popen(command_sequence[i],preexec_fn=os.setsid, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while script_process.poll() is None:
for output in iter(lambda: script_process.stdout.readline(), b''):
sys.stdout.writelines(output)
Non-blocking read on a subprocess.PIPE in python
Reliable non blocking reads from subprocess stdout
printing stdout in realtime from a subprocess that requires stdin
Constantly print Subprocess output while process is running
Is it possible to stream output from a python subprocess to a webpage in real time?
