read and process stdin output of another console application

Question

I want to manipulate the output of a console trace tool in a python script. So I want to read each line that the tool writes to stdout and maybe do something with that line and print it to the screen.

I have this script:

import sys

for l in sys.stdin:
    print l

and I start the trace tool like this:

qspy -c COM7 | my_script.py

the problem is when I hit enter the first time: nothing happens. When I hit enter again, I see all the output that was created by the tool but it also quits (the trace-tool quits on any character). What am I doing wrong?

i'm on windows.

EDIT:

I've tried all the suggestions, with little success:

first of all i created this little script to produce some output on stdout:

import time

while 1:
    time.sleep(1)
    print "test"

this does not work at all (no output):

import sys

for line in sys.stdin:
    print line,
    sys.stdout.flush()

this one also doesnt do anything:

import fileinput, sys

for line in fileinput.input():
    print line,
    sys.stdout.flush()

this one works, but only with the test-script, not with qspy:

import sys

for line in iter(sys.stdin.readline, ''):
    print line,
    sys.stdout.flush()

i did call the scripts like this:

test.py | python -u my_script.py

and did also:

set PYTHONUNBUFFERED=1

you are right, its useless. but also not part of the problem. — Dill, Jul 09 '12 at 17:38
http://stackoverflow.com/questions/1450393/how-do-you-read-from-stdin-in-python — dm03514, Jul 09 '12 at 17:38
@Dill -- So you get the same behavior without the while statement? If that's the case, you should edit and remove it. Leaving it there is just confusing. — mgilson, Jul 09 '12 at 17:51
try `python -u test.py | python my_script.py` where iter...readline and .flush are used in my_script. — jfs, Jul 10 '12 at 09:20

score 1 · Answer 1 · answered Jul 09 '12 at 17:54

1

Try to disable buffering:

$ qspy -c COM7 | python -u my_script.py

Also you could try to force reading line by line:

for line in iter(sys.stdin.readline, '')
    print line,

Check wether qspy has an option that manages buffering.

You could also set PYTHONUNBUFFERED environment variable.

answered Jul 09 '12 at 17:54

jfs

399,953
195
994
1,670

@Dill: if qspy is a Python script you could run it as: `python -u path/to/qspy` or set the environment variable and it should obey. – jfs Jul 10 '12 at 09:15
@Dill: if `qspy` uses stdio.h for IO and it doesn't override the buffering of its standard streams then you could try [`stdbuf`](http://www.pixelbeat.org/programming/stdio_buffering/stdbuf-man.html) to set line buffering of its stdout stream: `stdbuf -o L qspy -c COM7 | python ...`. Or run `qspy` from inside of `my_script.py` using `winpexpect` module. – jfs Jul 25 '12 at 06:09
comes time ill try this. what i did as a quick hack was to change the source of qspy to flush after each write to the logfile and open the logfile with python. i know that this isnt a good solution. – Dill Jul 31 '12 at 19:04

read and process stdin output of another console application

1 Answers1