question about pexpect in python

Question

I tried both pexpect and subprocess.Popen from python to call an external long term background process (this process use socket to communicate with external applications), with following details.

subprocess.Popen(launchcmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) This works fine. I do not need to do anything else. However, because I have to get the output immediately, I choose pexpect to avoid the pipe file buffer problem.
obj= pexpect.spawn(launchcmd, timeout=None) after launching external process, I use a separate thread to do "readline" to read the output of the launched process "obj", and everything is ok.
obj= pexpect.spawn(launchcmd, timeout=None) after launching external process, I did nothing further, i.e., just leave it there. Although, by using the "ps -e" command I can find the launched process, but the launched process seems blocked and cannot communicate in sockets with other applications.

OK. To be more specific, I put some sample code to formulate my question.

import subprocess
import pexpect
import os

t=1
while(True):
    if(t==1):
        background_process="./XXX.out"
        launchcmd = [background_process]
        #---option 3--------
        p=pexpect.spawn(launchcmd, timeout=None) # process launced, problem with socket.
        #---option 1--------
        p=subprocess.Popen(launchcmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) # process launced, everything fine
        t=0

Could anyone tell me what's wrong with the 3rd option? And if it is due to the fact that I did not use a separate thread to manipulate the output, why 1st option works with subprocess.popen? I suspect there is something wrong with pexpect to launch a process using socket, but I am not sure, especially considering option 2 works well.

I think you may be mistaking the difference between piping and buffers - a pipe is just a conduit, if it's buffered or not depends on the application thats writing to it (before you read). Readline is buffering -- until you get a newline. read() is buffered because it reads until it gets an EOF. The first example you provide should be fine, if there's data to read you'll get it, but if your launchcmd is buffering it won't be instant. — synthesizerpatel, Dec 07 '10 at 12:41
hi, synthesizerpatel, thank you for your comment. but I think it does not really help my question. — pepero, Dec 07 '10 at 14:02
@synthesizerpatel: on Linux, a pipe is buffered, by the kernel. This buffering is independent of the in-process buffering and so far, I haven't found a way of turning it off. — Fred Foo, Apr 15 '11 at 15:48
Can you explain what you mean with sockets? I cannot see them in your code. — pyfex, May 11 '11 at 14:25

score 1 · Answer 1 · edited May 23 '17 at 12:27

I think that you are making this too complicated.

Yes, it is a good idea to use a pty instead of a pipe to communicate with the background process because most applications recognize tty/pty devices and switch to using unbuffered output, (or at least line-buffered).

But why pexpect? Just use Python's pty module. First call openpty to get some filehandles and then use Popen to spawn the process. Example code is found in the following question (the answer with the green checkmark) Python Run a daemon sub-process & read stdout

question about pexpect in python

1 Answers1