Run a python script inside a running program

Question

I am running a python script that launches a executable called ./abc This executable enters inside of a program and waits for a command like so:

$./abc
abc >                      \\waits for a command here.

What I would like to do is to enter a couple of commands like:

$./abc
abc > read_blif alu.blif
abc > resyn2

What I have so far is as follows:

import os
from array import *

os.system('./abc')
for file in os.listdir("ccts/"):
    print 'read_blif ' + file + '\n'
    print 'resyn2\n'
    print 'print_stats\n'
    print 'if -K 6\n'
    print 'print_stats\n'
    print 'write_blif ' + file.split('.')[0] + 'mapped.blif\n'

This however will do the following:

abc >                 \\stays idle and waits until I ^C and then it prints
read ...blif
resyn2
...

It prints just to the terminal. How do I make it execute this inside the program and wait until it sees the next abc > to run the next command. Thanks

You are shadowing builtin `file` (for _Python_ 3.4 (probably _Python3_ altogether) it's OK, but I see that it's not the case here). Consider renaming it. What does the _abc_ executable actually do when you execute it from cmdline? You'll have to be more specific. — CristiFati, Dec 07 '16 at 21:17
Sorry for the vague description but all the answers bellow were correct. I just wanted to capture the terminal output of a program and write command to it. — Pika, Dec 07 '16 at 22:25

morsecodist · Answer 1 · 2016-12-07T21:24:35.433

I have done something similar using subprocess.

import subprocess

cmd = './command_to_execute'
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
result = pro.stdout.read()

This will execute the command specified by cmd, then reads the result into result. It will wait for there to be a result printed to the console before executing anything after the result assignment. I believe this might be what you want though your description was a bit vague.

score 1 · Answer 2 · answered Dec 07 '16 at 21:57

You may be looking for the pexpect module. Here is the basic example from pexpect's documentation

# This connects to the openbsd ftp site and
# downloads the recursive directory listing.
import pexpect
child = pexpect.spawn('ftp ftp.openbsd.org')
child.expect('Name .*: ')
child.sendline('anonymous')
child.expect('Password:')
child.sendline('noah@example.com')
child.expect('ftp> ')
child.sendline('lcd /tmp')

I think it will work the same way with abc >, if your OS is compatible with pexpect.

score 0 · Answer 3 · edited May 23 '17 at 12:13

If you want to pipe commands into the input of an executable, the easiest way would be to use the subprocess module. You can input into the stdin of the executable and get its output with Popen.communicate.

import subprocess

for f in os.listdir('ccts/'):
    p = subprocess.Popen('./abc', stdin=subprocess.PIPE,
                         stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    stdout, stderr = p.communicate('read_blif ' + f + '\n'
                                   'resyn2\n'
                                   'print_stats\n'
                                   'if -K 6\n'
                                   'print_stats\n'
                                   'write_blif ' + f.split('.')[0] + 'mapped.blif\n')
    # stdout will be what the program outputs and stderr will be any errors it outputs.

This will, however, close stdin of the subprocess everytime, but is the only way to reliable communicate without deadlocking. According to https://stackoverflow.com/a/28616322/5754656, you should use pexpect for an "interactive session-like" program. This is avoided here by having multiple subprocesses, assuming you can have different children run the program.

I assume that you only need the stdouts of print_stats, so you can do (as an example, you might want to handle errors):

import subprocess

def helper_open(cmd='./abc'):
    return subprocess.Popen(cmd, stdin=subprocess.PIPE,
                            stdout=subprocess.PIPE, stderr=subprocess.PIPE)

for f in in os.listdir('cts/'):
    helper_open().communicate('read_blif' + f + '\n'
                              'resyn2\n')
    stats, _ = helper_open().communicate('print_stats\n')
    # Do stuff with stats
    stats, _ = helper_open().communicate('if -K 6\nprint_stats\n')
    # Do more stuff with other stats
    helper_open().communicate('write_blif ' + f.split('.')[0] + 'mapped.blif\n')

Thanks for your answer. I am am implementing what you wrote for the first part. However when I try your second suggestion I get `ValueError: Cannot send input after starting communication`. I basically just implemented what you had in the first suggestion and did subprocess.Popen in the beginning of every for loop iteration and then took the terminal output and I parse that to get the stats. — Pika, Dec 07 '16 at 22:26
@Pika Sorry, I didn't realise that `Popen.communicate` is a one-time thing. If you can run the executable multiple times, my current edit will work. If not, use the `pexpect` answer if on Unix. — Artyer, Dec 07 '16 at 22:38
That's okay. Thanks a lot! I just do that parsing after I write the terminal output to a file and I just run the executable multiple times. — Pika, Dec 07 '16 at 22:48

score 0 · Answer 4 · answered Dec 07 '16 at 21:33

You need to spawn a new process using subprocess library and make two pipes one for stdin and one for stdout. Using this pipes ( that are represented in python as files) you can communicate with your process Here is an example:

 import subprocess

cmd = './full/path/to/your/abc/executable'
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, 
                       stdin=subprocess.PIPE)

pro.stdin.write("read_blif alu.blif \n")
pro.stdout.read(3)

You can use pro.communicate but I assume you need to take the output for every command you input. something like:

abc > command1
ouput1
abc > command2
output2 -part1
output2 -part2
output2 -part3

In this way I think the PIPE approach is more useful.

Use the dir function in python to find more info about the pro object and what methods and attributes are available dir(proc). Don't forget the help built in that will display the docstrings help(pro) or help(pro.stdin).

You are making a mistake when you run os.system this will run you program in the background an you won't have control over it. Maybe you would like to look into what input/output stream. More reading can be done here.

Run a python script inside a running program

4 Answers4