2

Sorry, I imagine this has to have been asked before, but for some reason I can't seem to find an answer.

I am trying to launch a long running child process such as an REPL (lein repl, python) and communicate with it via stdin/stdout. I have found many examples of how to do this when the parent only writes to the child once and then reads some input, but I am unable to figure out how to do this where I can write and read to/from the child process' stdin/stdout for an indefinite period like I would with a TCP socket.

Ideally I am trying to do this in python, but I wouldn't mind seeing how to do this in C or C++. I came across a library called pexpect (https://pexpect.readthedocs.org/en/latest/) which is close to what I'm looking for, but it seems to be geared for making sure the child process is running as expected or passing off control of the terminal to the new process, not bidirectional communication. I also noticed Python Twisted, but the docs are a bit dense and I still can't tell if it can do what I'm looking for

Edit: Here's some code I have right now

from subprocess import Popen, PIPE

p = Popen('lein repl 1>&2', shell=True, stderr=PIPE)

# p.communicate('(print "Hello, world!")')[0]

for line in iter(p.stderr.readline, b''):
  print('my-repl>> ' + line)
  # p.stderr.flush()


print 'End of script'

This code will launch the program and allow me to interact with it directly. When I add stdin=PIPE however, the program freezes. If I add stdin=PIPE and try to call p.communicate I get an error of

ValueError: I/O operation on closed file

Edit 2: Some working code. It reads 10 lines (the current number output by nREPL at startup), then writes a line, then reads two lines.

from subprocess import Popen, PIPE

# Whether to use stdout or stderr as the output stream
# for the child process
proc_use_stderr = False

# Process Variables
p = None
p_out = None
p_in = None
# Config
nrepl_num_start_lines = 11

if proc_use_stderr:
  p = Popen('lein repl 1>&2', shell=True, stderr=PIPE, stdin=PIPE)
  p_out = p.stderr
else:
  p = Popen(['lein', 'repl'], stdout=PIPE, stdin=PIPE)
  p_out = p.stdout

p_in = p.stdin

child_output_lines = []
for i in range(0, nrepl_num_start_lines):
  child_output_lines.append(p_out.readline().strip('\n'))

# Leads to: ValueError: I/O operation on closed file
# p.communicate('(print "Hello, repl!")\n')[0]
p_in.write('(print "Hello, repl!")\n')
p_in.flush()

print 'Starting readline'
child_output_lines.append(p_out.readline().strip('\n'))
child_output_lines.append(p_out.readline().strip('\n'))
print 'Ending readline'

for i in range(0, len(child_output_lines)):
  print i, ':', child_output_lines[i]

# TODO: Terminal gets all goofy after the process exits
# p.stderr.flush()
# p.stderr.close()
p_out.flush()
p_out.close()
p_in.flush()
p_in.close()
p.terminate()

print 'Summary:'
print 'Num Received Lines:', len(child_output_lines)
print 'Line List:', child_output_lines
print 'End of script'
Weebs
  • 435
  • 6
  • 17
  • 1
    better give an example. won't `subprocess.Popen` satisfy your requirement? – Jason Hu Jul 04 '15 at 16:07
  • Popen works when I am only sending data to the child process once, but not indefinitely. After I write once the child process exits – Weebs Jul 04 '15 at 16:08
  • @Weebs: not true, you can communicate through the life of the process using `subprocess.PIPE`. Maybe you should show your failing code? In C/C++ you would uses pipes as well, using `popen`, there is an example here: http://pubs.opengroup.org/onlinepubs/009695399/functions/popen.html – cdarke Jul 04 '15 at 16:19
  • possible duplicate of [working of Popen.communicate()](http://stackoverflow.com/questions/16768290/working-of-popen-communicate) – Quark Jul 04 '15 at 16:20
  • @Weebs you operated wrongly. i bet. i don't see any write? – Jason Hu Jul 04 '15 at 16:41
  • @Quark I added some sample code. If I call p.communicate like I did above (commented out) it seems as if the process terminates before it reaches the for loop (gives me a ValueError) – Weebs Jul 04 '15 at 16:54
  • Update: I have something working. Although once the script finishes executing, for some reason the characters I type into the terminal stop appearing. – Weebs Jul 04 '15 at 17:12

0 Answers0