Getting live output from running unix command in python

Question

I am using below code for running unix commands:

cmd = 'ls -l'
(status,output) = commands.getstatusoutput(cmd)
print output

But the problem is that it shows output only after the command completed, but i want to see the output printed as the execution progresses. ls -l is just dummy command, i am using some complex command in actual program.

Thanks!!

Answer 1

Since this is homework, here's what to do instead of the full solution:

1. Use the subprocess.Popen class to call the executable. Note that the constructor takes a named stdout argument, and take a look at subprocess.PIPE.

2. Read from the Popen object's STDOUT pipe in a separate thread to avoid dead locks. See the threading module.

3. Wait until the subprocess has finished (see Popen.wait).

4. Wait until the thread has finished processing the output (see Thread.join). Note that this may very well happen after the subprocess has finished.

If you need more help please describe your precise problem.

Answer 2

Unless there are simpler ways in Python which I'm not aware of, I believe you'll have to dig into the slightly more complex os.fork and os.pipe functions.

Basically, the idea is to fork your process, have the child execute your command, while having its standard output redirected to a pipe which will be read by the parent. You'll easily find examples of this kind of pattern.

Answer 3

Most programs will use block buffered output if they are not connected to a tty, so you need to run the program connected to a pty; the easiest way is to use pexpect:

for line in pexpect.spawn('command arg1 arg2'):
    print line