Python reading from stdin hangs when interacting with ruby code

Question

I was trying to put python and ruby codes into conversation, and I found the methods from this link (http://www.decalage.info/python/ruby_bridge)

I tried the last method, using stdin and stdout to pass information. I made some changes to the origin code so that it fits python 3.4, but I am not sure whether or not the code that I changed messed all the things up. My python program always hangs when reading from stdin, and nothing was printed. I am not familiar with stdin and stdout, so I am just wondering why this does not work.

Here are my ruby codes:

$stdin.set_encoding("utf-8:utf-8")
$stdout.set_encoding("utf-8:utf-8")

while cmd = $stdin.gets

    cmd.chop!
    if cmd == "exit"
        break
    else
        puts eval(cmd)
        puts "[end]"
        $stdout.flush

    end
end

I am not sure if it is possible to set internal encoding and external encoding like this. And here are my python codes:

from subprocess import Popen, PIPE, STDOUT

print("Launch slave process...")
slave = Popen(['ruby', 'slave.rb'], stdin=PIPE, stdout=PIPE, stderr=STDOUT)

while True:
    line = input("Enter expression or exit:")
    slave.stdin.write((line+'\n').encode('UTF-8'))
    result = []
    while True:
        if slave.poll() is not None:
            print("Slave has terminated.")
            exit()

        line = slave.stdout.readline().decode('UTF-8').rstrip()
        if line == "[end]":
            break
        result.append(line)
    print("result:")
    print("\n".join(result))

When I try to run the python script, input "3*4", and press enter, nothing shows until I broke the process manually with exit code 1 and KeyboardInterrupt Exception. I have been struggling with this problem for quite a long time and I don't know what goes wrong... Thanks in advance for any potential help!

score 2 · Accepted Answer · edited May 23 '17 at 11:43

The difference is that bufsize=-1 by default in Python 3.4 and therefore slave.stdin.write() does not send the line to the ruby subprocess immediately. A quick fix is to add slave.stdin.flush() call.

#!/usr/bin/env python3
from subprocess import Popen, PIPE

log = print
log("Launch slave process...")
with Popen(['ruby', 'slave.rb'], stdin=PIPE, stdout=PIPE, 
           bufsize=1, universal_newlines=True) as ruby:
    while True:
        line = input("Enter expression or exit:")
        # send request
        print(line, file=ruby.stdin, flush=True)
        # read reply
        result = []
        for line in ruby.stdout:
            line = line.rstrip('\n')
            if line == "[end]":
                break
            result.append(line)
        else: # no break, EOF
            log("Slave has terminated.")
            break
        log("result:" + "\n".join(result))

It uses universal_newlines=True to enable text mode. It uses locale.getpreferredencoding(False) to decode bytes. If you want to force utf-8 encoding regardless of locale settings then drop universal_newlines and wrap the pipes into io.TextIOWrapper(encoding="utf-8") (code example -- it also shows the proper exception handling for the pipes).

Python reading from stdin hangs when interacting with ruby code

1 Answers1