1

I've got this C program:

#include <stdio.h>
#include <Windows.h>

int main() {
    for (int i=0;; i++) {
        printf("%i\n",i);
        Sleep(100);
    }
    return 0;
}

And I have a Python script that tries to capture its output and do something with it:

from subprocess import Popen, PIPE

p = Popen("a.exe", stdout=PIPE, shell=True)
print p.stdout.readline()

... and it hangs on the last line without printing anything on a screen.

I've tried to solve this problem using the Python shell and found this:

>>> from test import *
>>> p.stdout.flush()
>>> p.stdout.readline()
'0\r\n'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
^CKeyboardInterrupt

It can actually read the output but only when I send KeyboardInterrupt. p.stdout.read(1) behaves the same way.

So, what's a working, correct way to do what I want?

Edit:
Ok, looks like it is impossible on Windows, see comments to first answer.

Anton Kapelyushok
  • 13
  • 1
  • 4

3 Answers3

0

The output is being buffered so you need to useiter(p.stdout.readline,"")

p = Popen("a.exe", stdout=PIPE)

for line in iter(p.stdout.readline,""):
     print line

Try flushing stdout from c if sys.stdout.flush() if is not working, as far as I know lines are block buffered when writing to a pipe:

int main() {
    for (int i=0;; i++) {
        printf("%i\n",i);
        Sleep(100);
        fflush(stdout);

    }
    return 0;
}
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321
  • Just saw the sleep command, that is causing the issue, try flushing stdout – Padraic Cunningham Mar 18 '15 at 21:27
  • fflush(stdout) works but I cant do this. Originally "a.exe" is a server and i havent access to its source code. sys.stdout.flush() doesnt work – Anton Kapelyushok Mar 18 '15 at 21:59
  • is there a `stdbuf` command on windows or a way to run the command redirecting stdout to stderr just to check? – Padraic Cunningham Mar 18 '15 at 22:00
  • I dont know. After quick googling i'd say no. But i found this: [Disable Buffering on Redirected Stdout Pipe win32 api c](http://stackoverflow.com/questions/3385427/disable-buffering-on-redirected-stdout-pipe-win32-api-c) and seems like it cant be done on Windows.. – Anton Kapelyushok Mar 18 '15 at 22:05
  • I had a look also, seems there is definitely not an equivalent command , you might be able to use pexpect, you need to somehow convince the app that is is writing to a console not a pipe – Padraic Cunningham Mar 18 '15 at 22:10
  • I really don't see a simple way of doing it from windows without using a pseudo-tty but how well it would work on windows I don't know. – Padraic Cunningham Mar 18 '15 at 22:17
  • There's no simple Windows equivalent of the Linux `LD_PRELOAD` code injection trick that `stdbuf` uses. You may be able to inject a DLL (via `OpenProcess`, `VirtualAllocEx`, `WriteProcessMemory`, `CreateRemoteThread` => `LoadLibrary`) that links with the target CRT to reopen the standard file descriptors and set no buffering or line buffering mode. I haven't tried this, however. – Eryk Sun Mar 19 '15 at 01:49
0

Rough example of something I did a long time ago.

process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while process.poll() is None:
    txt = process.stdout.read().decode("utf-8")
    result += txt

You can make your own StringIO which handles file io operations like read and write.

https://docs.python.org/3/library/io.html

I have not tested the code below!

import io
buffer = io.StringIO()
p = Popen(["a.exe"], stdout=buffer)
justengel
  • 6,132
  • 4
  • 26
  • 42
  • The problem is not with reading output. As I understood program buffers its output and you cant get any data until the buffer is full or the program is finished. Is there any way to force program to output data? p.stdout.flush() does nt work for some reason – Anton Kapelyushok Mar 18 '15 at 21:35
0

Run python in unbuffered mode:

python -u myprogram.py
whereswalden
  • 4,819
  • 3
  • 27
  • 41