Python, trying to get input from subprocess?

Question

main.py

import subprocess,sys
process = subprocess.Popen([sys.executable]+['example.py'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

while True:
    out = process.stdout.read(1)
    if not out:
        out=process.stderr.read(1)
    if out == '' and process.poll() != None:
        break
    if out != '':
        print out

example.py

f=raw_input('WHats your favorite animal')

Ok I am wondering how I can check for input in my main loop and be able to give some data to it. Right now my program freezes when I use raw_input.

Here is what I would like

while True:
    out = process.stdout.read(1)
    if not out:
        out=process.stderr.read(1)
    if out == '' and process.poll() != None:
        break
    #showing what i want
    if request_input==True:
        give_input('cat') #Give input to the raw_input
    #
    if out != '':
        print out

I dont know if there is a feature like this. If you need more explanation please comment.

Answer 1

This isn't really the best way to have interprocess communication, I recommend you either use the multiprocessing or threading libraries with something like either a Queue or PIPE for communication and synchronization.

a Queue would be simplest way for sharing data, one process would put in a value and another would get the values.

I've modified your original code so it works now, note that raw_input doesn't flush stdout as such if theres nothing there it will hang, http://code.activestate.com/lists/python-list/265749/ which is why your code was just waiting for stdout ...

THIS IS DANGEROUS, it can generate deadlocks, use at your own risk, try another method.

import sys
print 'Whats your favorite animal\n' #raw_input doesn't flush :( and we want to read in a whole line
sys.stdout.flush()
f = raw_input()
print f

and the corresponding main.py

import subprocess, sys, os
process = subprocess.Popen([sys.executable]+['example.py'],
     stdout = subprocess.PIPE, stderr = subprocess.PIPE, stdin = subprocess.PIPE)

while True:
    if process.poll() != None:
        break
    request_input = process.stdout.readline() # read line, this will wait until there's actually a line to read.
    if request_input == "Whats your favorite animal\n":
        print 'request: %s sending: %s' % (request_input, 'cat')
        process.stdin.write('cat\n')
       process.stdin.flush()

Answer 2

Edit: Seems like I missunderstood your question, sorry about that.

This is not as easy as you might think:

You have to use select to check if there is data available in stdin, if there is data, read it!

Minimal example:

STDIN, STDOUT = 0, 1

while not process.poll() is None:
    out = process.stdout.read(1)

    if not out:
        break

    try:
        fds = select.select([STDIN], [], [])[0]
    except select.error:
        pass
    else:
        if STDIN in fds:
            data = os.read(STDIN, 1024)

            if data_requested:
                process.stdin.write(data)

Information to select: http://docs.python.org/library/select.html#module-select, http://linux.die.net/man/2/select and http://en.wikipedia.org/wiki/Select_(Unix)

Not sure if this will work on windows, since Windows just supports select on sockets. Relevant question on SO: What is the best epoll/kqueue/select equvalient on Windows?

Answer 3

Try to use the multiprocessing module

from multiprocessing import Process, Value, Array

def f(n, a):
    n.value = 3.1415927
    for i in range(len(a)):
        a[i] = -a[i]

if __name__ == '__main__':
    num = Value('d', 0.0)
    arr = Array('i', range(10))

    p = Process(target=f, args=(num, arr))
    p.start()
    p.join()

    print num.value
    print arr[:]

Answer 4

Unless you insist on "raw python", the best is to use pexpect module. I've added an output to your example.py, otherwise it is no fun at all.

Here is example.py:

    f=raw_input('WHats your favorite animal')
    print f.upper()

Here is pexpect-example.py you are looking for:

    # see http://pexpect.sourceforge.net/pexpect.html
    import pexpect

    PY_SCRIPT = 'example.py'

    child = pexpect.spawn('python %s' % PY_SCRIPT)

    child.expect ('WHats your favorite animal')

    # comment out these three lines if you run unmodified example.py script
    child.setecho(False)
    child.sendline ('cat')
    print PY_SCRIPT, 'said:', child.readline()