Is there a way to capture stdout from this example?

Question

This is for the Windows environment.

Compile the testprocess.py (using pyinstaller) and place the resulting exe in a test folder.

In the same folder, run the ptest.py.

Testprocess.py starts and never ends, writing a number every 3 seconds to stdout.

ptest.py attempts to capture this output.

This code simulates a production issue that I would like to solve. Similar to what happens in production, the stdout is not released to the ptest.py until the testprocess terminates. In production, this process never stops, yet it posts important content to stdout.

Is there a method to accomplish this?

The attached code works fine, as long as the child process terminates.

## [testprocess.py]:

import time

x = 0

while True:
    print(x)
    time.sleep(3)
    x += 1


## [ptest.py]:

import os
import sys
import subprocess

def get_script_path():
    return os.path.dirname(os.path.realpath(sys.argv[0]))

start_dir = get_script_path()

cmd = [start_dir + os.sep + 'testprocess.exe']

proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', universal_newlines=True)

print('Subprocess started.')

capture = ""

s = proc.stdout.read(1)

print('Read Stdout')

while len(s) > 0:
    sys.stdout.write(s)
    sys.stdout.flush()
    capture += s
    s = proc.stdout.read(1)
    print(s)

print(capture)

sys.exit()

Would like to be able to capture the stdout of the child process while it is still running, not waiting until it terminates.

Have you tried [this](https://stackoverflow.com/a/18422264/12096138)? — ImranD, Oct 15 '19 at 15:03
Yes - it also seems to wait until the child process terminates to output. — user12197083, Oct 15 '19 at 15:09

SyntaxVoid · Answer 1 · 2019-10-15T15:17:18.710

0

It is possible, and it's easier than you think. Once the subprocess is started, you can continuously try to read from stdout and print it if there's something to print. You'll probably have to modify testprocess.py to flush itself (adding flush = True to the print statements).

p = subprocess.Popen(command, 
                     stdout = subprocess.PIPE, 
                     stderr = subprocess.STDOUT, 
                     encoding='utf-8', 
                     universal_newlines=True)

while True:
    line = p.stdout.readline()
    if line == "" and p.poll() is not None:
        break
    if line:
        print(line.strip(), flush = True)

Edit: If your command looks like python testprocess.py, you can skip having to add the flush = True's to your print statements by instead passing -u as a command option. The -u tells the python interpreter to operate in unbuffered mode.

But, I see your command is actually calling an exe file. You might need to figure out how to tell your compiler how to compile your program as unbuffered.

edited Oct 15 '19 at 15:17

answered Oct 15 '19 at 15:12

SyntaxVoid

2,501
2
15
23

Actually, I have tried this very code, but I get the same result - no output until the child process terminates. I just tried it again - same result. – user12197083 Oct 15 '19 at 15:17
Did you add `flush = True` to the print statements in `testprocess.py` and then recompile into your exe? – SyntaxVoid Oct 15 '19 at 15:17
Check this answer: https://stackoverflow.com/a/50682752/3308951, you might have to opt for the `sys.stdout.flush()` route. At this point I think your options are: write a `print` that you put in `testprocess.py` which prints, and then calls `sys.stdout.flush()` or figure out how to pass the unbuffered option to PyInstaller. I've found a few questions about the latter, but no solutions. – SyntaxVoid Oct 15 '19 at 15:21

Is there a way to capture stdout from this example?

1 Answers1