0

Basically I'm trying to automate some Linux installers (and other tasks) using the subprocess library (Popen).

In the past I've been able to open processes like this: 

    self.process = subprocess.Popen( self.executable,
                                stdout=subprocess.PIPE, 
                                stderr=subprocess.STDOUT,
                                universal_newlines=True, shell=True)
    output, cmdError = self.process.communicate()

I can then print output or cmdError for error messages and this works well for single processes or commands.

But when I need to interact with a subprocess and examine the output, it is very difficult, here is my code for doing this:

def ExecProcessWithAnswers(self):
    self.process = subprocess.Popen( self.executable,
                                stdout=subprocess.PIPE, 
                                stderr=subprocess.STDOUT,
                                stdin=subprocess.PIPE,
                                universal_newlines=True, shell=True)
    while self.process.poll() is None:
        print self.process.stdout.readline()

Basically the idea is that I would poll the output (stdout pipe) and then send commands when a certain input is requested from the installer (stdin).

I've tried flushing the buffer, using 3 different ways to read/iterate the output from stdout, but all of them just block or only give me a small fraction of the output text. On the other hand, if I use the communicate method, I get all of the text I expect, but it terminates the process. I've also had EOF errors and other random things.

I've read around: some guides say this is a bug in 2.6.x but it is still in 2.7.x - apparently the stdout is buffered and cannot be changed. I've tried many different ways of parsing the output from various threads here but I still can't get this to work on 2.7.X.

Surely someone must know how to interact with a subprocess? Is my only option here to use pexcept?

I can't really switch to Python 3.x.x within my environment. I was hoping this would be fairly straight forward :(

Cheers

Edit: I've also tried removing the different Pipes, writing to files, changing the buffer size on popen, disabling the shell and universal newlines, etc.

user2078888
  • 27
  • 2
  • 5
  • [This question](http://stackoverflow.com/questions/28753762/execute-script-on-snort-alert) presents 3 different ways to work around buffering that may occur. Note that only the first approach is available on Windows. – jedwards Mar 11 '15 at 01:22
  • Many thanks - I seem to have solved the issue for now using file descriptors with the os and pty libraries. There seems to be some strange race condition with the particular process I'm working with and pipes. I'll post the solution when I have a chance. – user2078888 Mar 11 '15 at 10:52
  • note: @jedwards's code examples are buggy ([I've left the comment there](http://stackoverflow.com/questions/28753762/execute-script-on-snort-alert#comment46226914_28929185)). – jfs Mar 11 '15 at 14:30
  • related: [Multiple inputs and outputs in python subprocess communicate](http://stackoverflow.com/q/28616018/4279) – jfs Mar 11 '15 at 14:30
  • Make sure to read: [Q: Why not just use a pipe (popen())?](http://pexpect.readthedocs.org/en/latest/FAQ.html#whynotpipe) – jfs Mar 11 '15 at 14:31
  • @J.F.Sebastian I'm running into the exact issue you described, the script works 100% perfectly until the subprocess ends and I can't detected EOF - it just hangs. Adding a timeout fixes the issue. I'm building a PIP library, thanks a lot guys! – user2078888 Mar 12 '15 at 12:27
  • @user2078888: [`pty` + `select()` might be too low level](http://stackoverflow.com/a/20509641/4279); [`pexpect` might be simpler to use and more portable](http://stackoverflow.com/a/23795689/4279). – jfs Mar 12 '15 at 12:37

0 Answers0