ruby open3 stdout and stdin how to interact

Question

sum.rb is very simple. You input two numbers and it returns the sum.

# sum.rb
puts "Enter number A"
a = gets.chomp
puts "Enter number B"
b = gets.chomp
puts "sum is #{a.to_i + b.to_i}"

robot.rb used Open3.popen3 to interact with sum.rb. Here's the code:

# robot.rb
require 'open3'

Open3.popen3('ruby sum.rb') do |stdin, stdout, stderr, wait_thr| 
  while line = stdout.gets
    if line == "Enter number A\n"
      stdin.write("10\n")
    elsif line == "Enter number B\n"
      stdin.write("30\n")
    else
      puts line
    end
  end
end

robot.rb failed to run. Seems it's stuck at sum.rb's gets.chomp.

Later I found out I have to write as following to make it work. You need to feed it with inputs before hand and in right sequence.

# robot_2.rb
require 'open3'

Open3.popen3('ruby sum.rb') do |stdin, stdout, stderr, wait_thr| 
  stdin.write("10\n")
  stdin.write("30\n")
  puts stdout.read
end

What confused me are:

1. robot_2.rb is not like interact with shell, it's more like feed what the shell needs, cause I just know. What if a program needs many inputs and we cannot predict the order?

2. I found out if STDOUT.flush been added after each puts in sum.rb, robot.rb could run. But in reality we cannot trust sum.rb's author could add STDOUT.flush, right?

Thanks for your time!

Answer 1

Finally figured out how to do this. Use write_nonblock and readpartial. The thing you have to be cautious of is that stdout.readpartial does exactly what it says, meaning you're going to have to aggregate the data and perform the gets yourself by looking for newlines.

require 'open3'
env = {"FOO"=>"BAR", "BAZ"=>nil}
options = {}
Open3.popen3(env, "cat", **options) {|stdin, stdout, stderr, wait_thr|
    stdin.write_nonblock("hello")

    puts stdout.readpartial(4096)
    # the magic 4096 is just a size of memory from this example:
    # https://apidock.com/ruby/IO/readpartial


    stdin.close
    stdout.close
    stderr.close
    wait_thr.join
}

For people who are looking for more generic interactivity (for example ssh interactions), you probably want to create separate threads for aggregating the stdout and triggering stdin.

require 'open3'
env = {"FOO"=>"BAR", "BAZ"=>nil}
options = {}
unprocessed_output = ""
Open3.popen3(env, "cat", **options) {|stdin, stdout, stderr, wait_thr|

    on_newline = ->(new_line) do
        puts "process said: #{new_line}"
        # close after a particular line
        stdin.close
        stdout.close
        stderr.close
    end

    Thread.new do
        while not stdout.closed? # FYI this check is probably close to useless/bad
            unprocessed_output += stdout.readpartial(4096)
            if unprocessed_output =~ /(.+)\n/
                # extract the line
                new_line = $1
                # remove the line from unprocessed_output
                unprocessed_output.sub!(/(.+)\n/,"")
                # run the on_newline
                on_newline[new_line]
            end

            # in theres no newline, this process will hang forever
            # (e.g. probably want to add a timeout)
        end
    end

    stdin.write_nonblock("hello\n")

    wait_thr.join
}

BTW this is not very threadsafe. This is just an unoptimized but functional solution I've found that will hopefully be improved in the future.

Answer 2

I played with @jeff-hykin's answer a bit. So, the main bit is to send the data from sum.rb in nonblocking mode, i.e. use STDOUT.write_nonblock:

# sum.rb
STDOUT.write_nonblock "Enter number A\n"
a = gets.chomp
STDOUT.write_nonblock "Enter number B\n"
b = gets.chomp
STDOUT.write_nonblock "sum is #{a.to_i + b.to_i}"

-- notice the \n in the STDOUT.write_nonblock calls. It separates the strings/lines, that are read with gets "get string" in robot.rb. Then robot.rb can stay the same. I would only add strip in the conditions:

# robot.rb
require 'open3'

Open3.popen3('ruby sum.rb') do |stdin, stdout, stderr, wait_thr|
  while line = stdout.gets
    puts "line: #{line}" # for debugging
    if line.strip == "Enter number A"
      stdin.write("10\n")
    elsif line.strip == "Enter number B"
      stdin.write("30\n")
    else
      puts line
    end
  end
end

My Ruby version is 3.0.2.