1

I'm new to python, and trying to run a exe software from python in windows. I wrote the following code:

from subprocess import STDOUT, Popen, PIPE

cmd=r'C:\Users\lenaq\Desktop\sep\WATv16\TLWMA-0.09.exe'

with open('test.log', 'w') as f:
p = subprocess.Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
for c in iter(lambda: p.stdout.read(1), ''):
    sys.stdout.write(c)
    f.write(c)

The exe program have some running errors, and I need to get the output of the program in order to fix the params file in order to prevent the errors.

the problem is that by using the above code I don't get the full output of the exe (when comparing to the os.system() command). the error message window of the exe pops out before the completion of the output writing, and I don't know where is the problem.

can you please help me...

Jean-François Fabre
  • 137,073
  • 23
  • 153
  • 219
Hodaya Beer
  • 25
  • 5

1 Answers1

3

stderr=PIPE redirects the error stream to p.stderr, and you're not reading that (note that using p.communicate allows to get both stream results, but reading them separately can lead to deadlocks).

Anyway, if you don't care about merging both out & err streams, you could change that to:

stderr=STDOUT

so both out & err use the same stream p.stdout

Also: don't use shell=True, you don't need it.

If that doesn't fix it in your case, it means that the underlying program crashed while not flushing its output. Output flush works differently when output is not redirected, which may explain why you get more output when running it without redirection with os.system (more about this issue: forcing a program to flush its standard output when redirected)

One lead yet to be explored would be to use winpty which is an equivalent of unbuffer on Windows: What is the equivalent of unbuffer program on Windows?. Something like:

cmd = ["winpty.exe","-Xallow-non-tty","-Xplain","TLWMA-0.09.exe"]
p = subprocess.Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=STDOUT)
Jean-François Fabre
  • 137,073
  • 23
  • 153
  • 219
  • I'm sorry, I tried this but it didn't solve the problem.. any other ideas?... the output I get from the p.stdout is still not complete.. – Hodaya Beer Sep 27 '17 at 09:12
  • the problem is probably because the executable doesn't flush the output unless it's not redirected. That's a real pain. – Jean-François Fabre Sep 27 '17 at 09:25
  • you're right. but do u know how to fix it in python? – Hodaya Beer Sep 27 '17 at 09:59
  • I've asked a question here: https://stackoverflow.com/posts/comments/79846673?noredirect=1 because I'm also interested (windows-specific) and didn't get a straight answer. Sounds tough. – Jean-François Fabre Sep 27 '17 at 11:55
  • I tried to run the command `winpty.exe -Xallow-non-tty -Xplain TLWMA-0.09.exe | receive_unbuffered_output.exe` in the windows 2009 command prompt but I get this error: `'receive_unbuffered_output.exe' is not recognized as an internal or external com mand, operable program or batch file.` do you know what's the problem? – Hodaya Beer Oct 01 '17 at 08:33
  • of course, this is just an example. run the winpty command (without the pipe redirection) in your python program. see my edit – Jean-François Fabre Oct 01 '17 at 08:35
  • I doubt the reliability of polling the console, as winpty does, especially with the aggressive updates to the console in Windows 10. It's unlike a real server/client pty. But sometimes we just have to use what's available, even if it's dubious. – Eryk Sun Oct 07 '17 at 08:28
  • Interestingly, Microsoft added a ConDrv console device in Windows 8, which possibly could be used like a pty -- if its I/O control codes were documented or reverse engineered. The server (e.g. conhost.exe) opens "\Device\ConDrv" and clients open "\Device\ConDrv\Server" and subsequently "Connect" (for functions like `GetConsoleMode`, etc), "Input" (for `ReadConsole`, `ReadFile`), and "Output" (for `WriteConsole`, `WriteFile`). The new Linux subsystem also opens "\Device\ConDrv\KernelConnect" in the "System" process. – Eryk Sun Oct 07 '17 at 08:30
  • Microsoft is probably trying to catch up with linux, providing linux like shell as a standard as I heard but never tested personally – Jean-François Fabre Oct 07 '17 at 11:35