0

I'm experiencing an issue where a call to proc.communicate() still hangs even after calling proc.terminate().

I create tasks to be run in the background with the call

import subprocess as sub
p = sub.Popen(command, stdout=sub.PIPE, stderr=sub.PIPE, shell=False)

At the completion of the script, I call terminate(), communicate(), and wait() to gather information from the process.

p.terminate()
errorcode = p.wait()
(pout, perr) = p.communicate()

The script hangs at the call to communicate. I'd assumed that any call to communicate that follows a call to terminate would return immediately. Is there any reason why this would fail?

Edit: I'm using this method because the command is really a tight loop that won't terminate on its own. I'd like to use p.terminate() to do that, and then see what the stdout and stderr has to offer.

llvmstacker
  • 9
  • 3
  • 3
    Of course, you just terminated it, why don't you just call communicate? – Padraic Cunningham Jul 07 '16 at 22:04
  • if `p.wait()` returns but `p.communicate()` "hangs" then it may be related to [Python subprocess .check_call vs .check_output](http://stackoverflow.com/q/36169571/4279) (`check_call()` calls `.wait()` internally, `check_output()` calls `.communicate()` internally). Unrelated: No need to call `p.wait()` explicitly: `p.wait()` is called internally anyway and therefore you could use `p.returncode` after `p.communicate()` returns. In general, it is a bad practice to call `p.wait()` before `p.communicate()` it may deadlock a live process (it doesn't apply in this case -- the process is dead). – jfs Jul 07 '16 at 22:38
  • @PadraicCunningham: it is not so simple e.g., what if the child process doesn't exit until you terminate it explicitly. Another case: if the child process may spawn its own children then `p.communicate()` may be delayed; read my answer to [the linked question](http://stackoverflow.com/q/36169571/4279). – jfs Jul 07 '16 at 22:40
  • @J.F.Sebastian, yes valid points, that is why I asked *why don't you just call communicate?* As in, is there a specific reason. – Padraic Cunningham Jul 07 '16 at 22:50
  • The process being called in "command" does not spawn any children, and spins in a tight loop until it is terminated by something else. I can't just call communicate() because that will wait for the program to terminate on its own, which it will never do. – llvmstacker Jul 07 '16 at 23:43
  • @llvmstacker: how do you know that *"The script hangs at the call to communicate"* and not `p.wait()`? If there are no grandchildren processes then `p.communicate()` should be instantaneous for a reaped process (==`p.wait()` has returned). What is your exact Python version, OS, available RAM? – jfs Jul 08 '16 at 00:52
  • p.wait() returns immediately for sure. I get a returncode. I'm running Python 2.7 on 64-bit Windows 7 with 4 GB of RAM. – llvmstacker Jul 08 '16 at 15:24
  • Just regular old print statements reveal that the communicate() call is the one that does not return – llvmstacker Jul 08 '16 at 18:49

1 Answers1

1

You don't need the first two statements. Just call communicate().

Athena
  • 3,200
  • 3
  • 27
  • 35