Input python commands on the python command line

Question

The title to this may be confusing, but basically I want to be able to do the following:

import subprocess

subprocess.call(["python"])
subprocess.call(["import", "antigravity"])
subprocess.check_call(["print","\"This is so meta\" "])
subprocess.call(["exit()"])

Expected behavior would be that it would open up a python terminal session, then open up xkcd comic 353, print 'this is so meta' to the command line, and finally exit the python command line.

Basically, I want to be able to open a python session, and run commands in it from my python script. I also want to be able to check the output of commands I run in the script. Is this possible? and if so, what library do I need to be using? Will subprocess do this?

So, looks like you want your own kind of commands for the python? Am I right? — Nabin, Mar 15 '14 at 10:59
yeah, I want to be able to call my own commands in python. From a separate python script. — Derek Halden, Mar 15 '14 at 10:59
The python script that I want to run would be the commands listed above. I want to be able to run, and implement commands in, the "python command line" from a python script like the one I wrote in the question. — Derek Halden, Mar 15 '14 at 11:08
See this: http://stackoverflow.com/questions/22131980/python-scrapy-how-to-code-the-parameter-instead-of-using-cmd-use-custom-code-in — Nabin, Mar 15 '14 at 11:09
And perhaps this is the exact one: http://stackoverflow.com/questions/450285/executing-command-line-programs-from-within-python — Nabin, Mar 15 '14 at 11:14
Uhhh, not quite. My question is specifically about running commands inside python's command line feature. So, if you run `$ python` in your terminal, it will give you a kind of python command line. I want to be able to use subprocess, to pass python code into that. — Derek Halden, Mar 15 '14 at 11:26
I think you want to redirect stdin when you start the process. This will give you a file-like handle. Everything you write to that handle will be sent to the command line. If you redirect stdout too, everything the command line outputs to the "screen" will be written to that second handle. — Basic, Mar 15 '14 at 11:32
@Basic, Yeah, that sounds like it would work. How would I do that? — Derek Halden, Mar 15 '14 at 11:34

score 3 · Accepted Answer · answered Mar 15 '14 at 11:34

3

Something like this...

import subprocess
proc = subprocess.Popen(
    'python',stdout=subprocess.PIPE,
    stdin=subprocess.PIPE)
proc.stdin.write('import antigravity\n')
proc.stdin.write('print "something"\n')
proc.stdin.close()
result = proc.stdout.read()
print result

So we're creating a process and telling it that input will come from stdin (like someone typing). We then write anything we like to that and read the response from stdout (what would normally be printed to the screen)

answered Mar 15 '14 at 11:34

Basic

26,321
24
115
201

this solution suffers from [buffering issue](http://pexpect.readthedocs.org/en/latest/FAQ.html#whynotpipe) – jfs Mar 16 '14 at 05:47
@J.F.Sebastian Well... A) The OP was asking about input which _doesn't_ have a buffering issue and B) there are other answers on this question which have different approaches - so I'm not entirely sure what you're proposing I should do? Change mine to mimic someone else's answer? Seems a little like point stealing... – Basic Mar 16 '14 at 12:28
@Basic: A) Use then `communicate()` if you think OPs input is fixed B) *"What to do"*: Highlight possible issues with your solution. A simple warning with [the link](http://pexpect.readthedocs.org/en/latest/FAQ.html#whynotpipe) that explains it in more detail would be enough. – jfs Mar 16 '14 at 13:53

unutbu · Answer 2 · 2014-03-17T09:12:49.800

You could also use the code module:

import code
console = code.InteractiveConsole()
console.push('import antigravity')
console.push('print "something"')

If for some reason you wish to run this in a subprocess, then you could use the multiprocessing module:

import code
import multiprocessing as mp

def work():
    console = code.InteractiveConsole()
    console.push('import antigravity')
    console.push('print "something"')

if __name__ == '__main__':    
    proc = mp.Process(target=work)
    proc.start()
    proc.join()

To redirect stdout to a variable:

import code
import sys

class MyStream(object):
    def __init__(self, target):
        self.target = target
        self.value = None
    def write(self, s):
        if s.strip():
            self.value = s

sys.stdout = stream = MyStream(sys.stdout)
console = code.InteractiveConsole(locals=locals())
console.push('import antigravity')
console.push('print "something"')
sys.__stdout__.write('output: {}\n'.format(stream.value))

prints

output: something

Note that the console's sys.stdout has been redirected to MyStream(sys.stdout). It prints nothing, but stores the last string in self.value. To print to the string you could use sys.__stdout__.write (note the underscores).

Is there a way to use this to redirect output to variables in my code? — Derek Halden, Mar 15 '14 at 14:19

Roberto · Answer 3 · 2014-03-15T20:06:30.467

2

If you need to communicate with process you should use communicate() method instead stdin.write() otherwise you can find some no desirable effects.

Warning Use communicate() rather than .stdin.write, .stdout.read or .stderr.read to avoid deadlocks due to any of the other OS pipe buffers filling up and blocking the child process.

Source: http://docs.python.org/2/library/subprocess.html#popen-objects

from subprocess import PIPE, STDOUT, Popen

e = Popen(["/usr/local/bin/python3"], stdout = PIPE, stdin = PIPE, stderr = STDOUT, shell = False)

out, err = e.communicate(b"""
import sys
print('Interactive python version: %s' % str(sys.version))
sys.exit(4)
""")
e.wait()
print ('Exit code', e.returncode)
print ('Output', out)

edited Mar 15 '14 at 20:06

answered Mar 15 '14 at 11:42

Roberto

8,586
3
42
53

+1 for `.communicate()`. It is appropriate if the input doesn't depend on the output. btw, you don't need to call `e.wait()`; `.communicate()` already waits for the process to finish i.e., `e.returncode is not None` after it returns. For readability, you could use `from subprocess import Popen, PIPE, STDOUT` and use triple quotes to input the commands on multiple lines instead of `\n`. Also, `shell=False` and `[]` around a single argument are redundant. – jfs Mar 15 '14 at 13:39
@J.F. Sebastion: What if the input does depend on the output? – Derek Halden Mar 15 '14 at 13:52
@DerekHalden: then `.communicate()` is not appropriate. And you might have buffering issues that may cause a deadlock. See [Q: Why not just use a pipe (popen())?](http://pexpect.readthedocs.org/en/latest/FAQ.html#whynotpipe) – jfs Mar 15 '14 at 13:53
What would you use instead? – Derek Halden Mar 15 '14 at 13:54
@DerekHalden: In general, `pexpect` module. In your particular case, I 'd try [@unutbu's answer](http://stackoverflow.com/a/22423334/4279) first – jfs Mar 15 '14 at 13:56

Input python commands on the python command line

3 Answers3