4

I have a simple code which works fine on Win 2003:

proc = subprocess.Popen('<some python script which runs another process>', stdout = subprocess.PIPE, stderr = subprocess.PIPE, stdin = subprocess.PIPE)
out = proc.communicate()[0]

But on Windows 8 this part; out = proc.communicate()[0], hangs.

Have anybody seeen this issue?

  • I've checked that process is really ternimated (PID is absent when child process has been started)
  • It's also a problem to make proc.stdout.readlines(), it hangs too. How to check that stdout has EOF?
  • When I stop child process proc.communicate() works fine.

Here is the simplest example:

import subprocess
proc = subprocess.Popen([sys.executable, "D:\\test.py"], stdout = subprocess.PIPE)
print 'PID', proc.pid #When this PID is printed I see in the taskbr that process is already finished
print 'Output', proc.communicate() # but this part is hangs

And code od test.py:

import os, time
from subprocess import Popen, PIPE

CREATE_NEW_PROCESS_GROUP = 0x00000200  # note: could get it from subprocess
DETACHED_PROCESS = 0x00000008          # 0x8 | 0x200 == 0x208


p = Popen("start /B notepad", shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE,
          creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)
print 'Done'
exit()

Is it valid scenario?

Yulia
  • 43
  • 5
  • Two things, first try to run the command manually from windows cmd to be sure that the command works fine. Second have you try to run it with the `shell=True` flag? – jvallver May 13 '13 at 14:11
  • Command works fine from cmd, shell flag isn't important - is doesn't work bith both flags. Issue is in child process - while it's running - I can't get output from parent, although parent process is terminated. – Yulia May 14 '13 at 07:54
  • One think you can do for debug your program is use `proc.stdout.readline()` function in a while loop and print the output lines to see the time at which the process runs aground. – jvallver May 14 '13 at 08:02
  • Is the process stdout very long (more than 64kb)?, because I had problems in windows using `subprocess.PIPE` when process stdout is longer than 64kb. – jvallver May 14 '13 at 08:04
  • I've used proc.stdout.readline(). Output is short ~10 lines. I see all output of parent process and when it's finished (PID of process is disapeered from task manager), I see just empty line and realine hangs (also as readlines). Looks like child process inherits output of parent and locks it – Yulia May 14 '13 at 08:13
  • Ok, another think to do is redirect `stderr` to `stdout` using `stderr=subprocess.STDOUT` flag and then try to run another time the `proc.stdout.readline()` while loop. Then you can see if there is some message in the `stderr` output. – jvallver May 14 '13 at 08:24
  • 1
    I have seen it too... Note: The process called by `subprocess.Popen` from the parentmost script need not be a python script. It can as well be a batch file, which uses `start some-process.exe` to start a background process. – anishsane May 14 '13 at 08:29
  • try `os.system("start /B notepad")` – jvallver May 14 '13 at 08:37
  • running in background doesn't help too – Yulia May 14 '13 at 08:56
  • to anishsane: do you have information why it shouldn'y be a python? I have more complecate logic, and it based on python scripts only – Yulia May 14 '13 at 12:37

1 Answers1

2

To allow .communicate() to return without waiting for the grandchild (notepad) to exit, you could try in test.py:

import sys
from subprocess import Popen, PIPE

CREATE_NEW_PROCESS_GROUP = 0x00000200
DETACHED_PROCESS = 0x00000008

p = Popen('grandchild', stdin=PIPE, stdout=PIPE, stderr=PIPE,
          creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP)

See Popen waiting for child process even when the immediate child has terminated.

Community
  • 1
  • 1
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • I've made changes in test.py. It looks like yours ('grandchild' = 'C://Windows/System32/notepad.exe'). When I run test.py separetely, whithout p.communicate() process is finished even when notepad is running. But when I run test.py as subprocess in main file - .communicate() still hangs – Yulia May 14 '13 at 12:58
  • to avoid ambiguity, could you add the exact code of modified test.py to the question. What happens if grandchild="start notepad" (+shell=True)? – jfs May 14 '13 at 13:17
  • 2
    It works when I don't use stdout into PIPE: p = Popen('grandchild', close_fds = True, creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP). So, this solution suites me. Thank for your annswer! – Yulia May 15 '13 at 09:54