python interactive subprocess communicate

Question

I am trying to learn how to write interactive subprocess communication.

I need to read stdout from and write to stdin continually, below is my code, it sort of "works" but I am not sure if I am doing it right (it's very hacked code)

Assuming I have a script called app.py as following

import logging
import random

def app():
    number1 = random.randint(1,100)
    number2 = random.randint(200,500)
    logging.info("number1: %s, number2: %s", number1, number2)
    ans = input("enter sum of {} and {}: ".format(number1, number2))
    logging.info("received answer: %s", ans)
    try:
        if int(ans) != number1+number2:
            raise ValueError
        logging.info("{} is the correct answer".format(ans))
    except (ValueError,TypeError):
        logging.info("{} is incorrect answer".format(ans))

def main():
    logging.basicConfig(level=logging.DEBUG, filename='log.log')
    for x in range(10):
        app()

if __name__ == '__main__':
    main()

to interactive with above script (app.py) I have some very ugly code

import queue
import time
import threading
import subprocess
import os
import pty
import re

class ReadStdout(object):
    def __init__(self):
        self.queue = queue.Queue()
        self._buffer_ = []

    def timer(self, timeout=0.1):
        buffer_size = 0
        while True:
            if len(self._buffer_) > buffer_size:
                buffer_size = len(self._buffer_)
            time.sleep(timeout)
            if len(self._buffer_) == buffer_size and buffer_size!=0:
                self.queue.put(''.join(self._buffer_))
                self._buffer_ = []
                buffer_size = 0

    def read(self, fd):
        while True:
            self._buffer_.append(fd.read(1))

    def run(self):
        timer = threading.Thread(target=self.timer)
        timer.start()
        master, slave = pty.openpty()
        p = subprocess.Popen(['python', 'app.py'], stdout=slave, stderr=slave, stdin=subprocess.PIPE, close_fds=True)
        stdout = os.fdopen(master)
        read_thread = threading.Thread(target=self.read, args=(stdout,))
        read_thread.start()
        while True:
            if self.queue.empty():
                time.sleep(0.1)
                continue
            msg = self.queue.get()
            digits = (re.findall('(\d+)', msg))
            ans = (int(digits[0])+int(digits[1]))
            print("got message: {} result: {}".format(msg, ans))
            p.stdin.write(b"%d\n" %ans)
            p.stdin.flush()

if __name__ == '__main__':
    x = ReadStdout()
    x.run()

I don't feel I am doing it the right way. what's the correct way to interactive with another script (I need stdout, not just blind write to stdin)

Thanks

Possible duplicate of [Using Python's subprocess and Popen in one script to run another Python script which requires user interaction (by raw\_input)](https://stackoverflow.com/questions/12980148/using-pythons-subprocess-and-popen-in-one-script-to-run-another-python-script-w) — Yaroslav Surzhikov, Oct 15 '17 at 00:46
it's different. I know how to send stdin to the process. but I need to read the stdout first, then based on the result, write to stdin. the thread above do not worry stdout, it just blind feed in the stdin. — Rui Li, Oct 16 '17 at 19:35
I can also read stdout, but I am not sure if I'm reading it correctly. since readline() will not work because input() example do not send line break. I can read(size) but I don't know that is the "size" I need to read. assuming I don't really know the size of "input()" string size. — Rui Li, Oct 16 '17 at 19:42
This is Python 3, right? None of your syntax indicates it unambiguously, but I think you're relying on a subtlety of its `io` module in `app.py`... — Davis Herring, Oct 17 '17 at 07:49

Yaroslav Surzhikov · Answer 1 · 2017-10-17T08:19:59.580

This code will work with your app.py, so you can get base logic of interaction from it. Also, i would suggest you to look into pexpect module. In any case - you MUST know what to expect from running program and how it's input lines endswith. Or you could implement some timeout while reading line, so it would be possible to raise exception if something went not as expected.

import subprocess
from functools import partial

child = subprocess.Popen(['app.py'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True)

# iterate while child is not terminated
while child.poll() is None:
    line = ''
    # read stdout character by character until a colon appears
    for c in iter(partial(child.stdout.read, 1), ''):
        if c == ':':
            break
        line += c
    if "enter sum" in line:
        numbers = filter(str.isdigit, line.split())
        numbers = list(map(int, numbers))
        child.stdin.write("{0}\n".format(sum(numbers)))

So it seems if I don’t know what to expect, the only way is to have a timeout. There’s no better built-in way to work with it. Thank you for the help. — Rui Li, Oct 18 '17 at 01:31

python interactive subprocess communicate

1 Answers1