1

Very puzzled by this. When using Popen, if only using stdout or stderr the following code works:

def run(self):
    self.externalProcess = subprocess.Popen(['./external_process.out 1>&2'], shell=True, stderr=subprocess.PIPE)
    while self.externalBinary.poll() is None:
        print('Still running')
    print('Done running')

I'm using the stderr file descriptor because I'm monitoring the process output real time in a GUI and stderr is unbuffered. See my other question for more background on that mess: Python capture stdout from subprocess line by line

The problem I'm having here is that as soon as I add stdin to allow user input to be passed to the external process it starts acting as though stdout is buffered again:

def run(self):
    self.externalProcess = subprocess.Popen(['./external_process.out 1>&2'], shell=True, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
    while self.externalBinary.poll() is None:
        print('Still running')
    print('Done running')

My question is why does stdin appear to be affecting stdout and stderr?

Community
  • 1
  • 1
anderspitman
  • 9,230
  • 10
  • 40
  • 61
  • What exactly are the symptoms of your problem? Something blocked that shouldn't be? – jwd Oct 03 '11 at 18:33
  • Also, working code would be appreciated - your examples do not run as written. What is `stdout=STDERR`, for example? – jwd Oct 03 '11 at 18:33
  • Yeah I noticed that there is no such thing as subprocess.STDERR. Should be fixed now. The symptoms are that I have a textbox in my wxPython app that displays the output a line at a time. When I add the stdin argument it only displays a few lines that stops. When I remove it it continues displaying the output from my external process, which is a C program. – anderspitman Oct 03 '11 at 18:39
  • Where do you actually send input to the child process? I don't see that happening in your second example... – jwd Oct 03 '11 at 18:43
  • I didn't include it because I see the issue as soon as I add the argument. I don't even need to try using stdin. After reading the link below I think my problem is that I'm deadlocking due to caching, so I'm working on getting around that. I don't need to use stdin until I close down the child process (I'm just sending a newline to simulate the enter key being pressed which is how that program exits cleanly). – anderspitman Oct 03 '11 at 20:25

1 Answers1

1

It seems like you might be trying to write data to the subprocess (via STDIN) as well as read its output (via STDOUT) interactively.

As mentioned in the answer to this SO question, that is a no-no!

You could use Popen.communicate, but that is a blocking call which will wait for all communication to finish (you can't poll as you do in your example)

Community
  • 1
  • 1
jwd
  • 10,837
  • 3
  • 43
  • 67
  • Good link. I didn't understand the deadlocking issue before. I'm trying to get communicate working but it's not so far. See my comment above. – anderspitman Oct 03 '11 at 20:26
  • @tanders12: What problems are you having with `communicate()` ? – jwd Oct 03 '11 at 22:34
  • communicate() appears to buffer everything until the child exits. I need it line buffered or unbuffered. I'm trying to accomplish this without changing the code in the child. Right now I'm attempting to make pexpect work and it looks promising. – anderspitman Oct 03 '11 at 22:42