9

I am using python 2.5 on Windows. I wish to interact with a console process via Popen. I currently have this small snippet of code:

p = Popen( ["console_app.exe"], stdin=PIPE, stdout=PIPE )
# issue command 1...
p.stdin.write( 'command1\n' )
result1 = p.stdout.read() # <---- we never return here
# issue command 2...
p.stdin.write( 'command2\n' )
result2 = p.stdout.read()

I can write to stdin but can not read from stdout. Have I missed a step? I don't want to use p.communicate( "command" )[0] as it terminates the process and I need to interact with the process dynamically over time.

Thanks in advance.

TravisHendrickson
  • 51
  • 5
QAZ
  • 4,870
  • 6
  • 36
  • 50
  • Your code seems correct. Does the console_app work ok when you execute from the console? What does it return for command1? – luc Jul 14 '09 at 13:04
  • yes console_app works normally when run in cmd.exe It just outputs some numbers based on the input provided (and sometimes strings) – QAZ Jul 14 '09 at 13:13

5 Answers5

8

Your problem here is that you are trying to control an interactive application.

stdout.read() will continue reading until it has reached the end of the stream, file or pipe. Unfortunately, in case of an interactive program, the pipe is only closed then whe program exits; which is never, if the command you sent it was anything other than "quit".

You will have to revert to reading the output of the subprocess line-by-line using stdout.readline(), and you'd better have a way to tell when the program is ready to accept a command, and when the command you issued to the program is finished and you can supply a new one. In case of a program like cmd.exe, even readline() won't suffice as the line that indicates a new command can be sent is not terminated by a newline, so will have to analyze the output byte-by-byte. Here's a sample script that runs cmd.exe, looks for the prompt, then issues a dir and then an exit:

from subprocess import *
import re

class InteractiveCommand:
    def __init__(self, process, prompt):
        self.process = process
        self.prompt  = prompt
        self.output  = ""
        self.wait_for_prompt()

    def wait_for_prompt(self):
        while not self.prompt.search(self.output):
            c = self.process.stdout.read(1)
            if c == "":
                break
            self.output += c

        # Now we're at a prompt; clear the output buffer and return its contents
        tmp = self.output
        self.output = ""
        return tmp

    def command(self, command):
        self.process.stdin.write(command + "\n")
        return self.wait_for_prompt()

p      = Popen( ["cmd.exe"], stdin=PIPE, stdout=PIPE )
prompt = re.compile(r"^C:\\.*>", re.M)
cmd    = InteractiveCommand(p, prompt)

listing = cmd.command("dir")
cmd.command("exit")

print listing

If the timing isn't important, and interactivity for a user isn't required, it can be a lot simpler just to batch up the calls:

from subprocess import *

p = Popen( ["cmd.exe"], stdin=PIPE, stdout=PIPE )
p.stdin.write("dir\n")
p.stdin.write("exit\n")

print p.stdout.read()
rix0rrr
  • 9,856
  • 5
  • 45
  • 48
  • 1
    It won't gonna work on windows as .read(1) is blocking operation – Piotr Czapla Jul 15 '09 at 18:19
  • I know, that's why after every byte the script checks if the prompt is available and writes the new command then before reading any more bytes. Did you even try the script? It works. – rix0rrr Jul 15 '09 at 22:15
  • Right, I didn't spot that at first glance. +1 for nice expect implementation. – Piotr Czapla Jul 20 '09 at 12:03
  • You can exclude the prompts in the output too by doing this: "\n".join(commandOutput.split('\r\n')[1:-1]) – Tjaart Feb 16 '12 at 12:46
  • 3
    In case of `cmd.exe` you can customize the prompt to make processing easier. For example using `cmd.exe /k prompt $g$g$g$_` will change the prompt to `>>>\n`, so that you can remove pattern matching and use readline() again. – yurez Nov 13 '12 at 20:37
  • @yurez This should be a seperate answer to the question – McSebi Oct 25 '21 at 12:51
2

Have you tried to force windows end lines? i.e.

p.stdin.write( 'command1 \r\n' )
p.stdout.readline()

UPDATE:

I've just checked the solution on windows cmd.exe and it works with readline(). But it has one problem Popen's stdout.readline blocks. So if the app will ever return something without endline your app will stuck forever.

But there is a work around for that check out: http://code.activestate.com/recipes/440554/

Piotr Czapla
  • 25,734
  • 24
  • 99
  • 122
0

I think you might want to try to use readline() instead?

Edit: sorry, misunderstoud.

Maybe this question can help you?

Community
  • 1
  • 1
Robert Massa
  • 4,345
  • 1
  • 32
  • 44
  • readline() just hangs too. I can confirm that the console app does output data which should be read in correctly on the python side. – QAZ Jul 14 '09 at 11:56
0

Is it possible that the console app is buffering its output in some way so that it is only being sent to stdout when the pipe is closed? If you have access to the code for the console app, maybe sticking a flush after a batch of output data might help?

Alternatively, is it actually writing to stderr and instead of stdout for some reason?

Just looked at your code again and thought of something else, I see you're sending in "command\n". Could the console app be simply waiting for a carriage return character instead of a new line? Maybe the console app is waiting for you to submit the command before it produces any output.

Midpoint
  • 213
  • 3
  • 7
  • Thanks, good suggestions. I dont have access to the console app's source code unfortunately. Confirmed its writing to stdout and not stderr. – QAZ Jul 14 '09 at 12:21
0

Had the exact same problem here. I dug into DrPython source code and stole wx.Execute() solution, which is working fine, especially if your script is already using wx. I never found correct solution on windows platform though...

Wojciech Bederski
  • 3,852
  • 1
  • 25
  • 28