Subprocess Deadlock

Question

I have a shell script which I need to invoke in the Python program.

from subprocess import *
p=Popen('some_script',stdin=PIPE)
p.communicate('S')

But after the communicate() command is executed the process goes for a deadlock. The shell script has not been written by me and I cannot modify it. I just need to kill the process after the communicate() method is executed, but the program should not exit. Please help me in this regard.

are you sure that `some_script` reads its input from stdin and not from the terminal/console directly? — jfs, Mar 18 '16 at 15:42
Yes, 'some_script' reads input from terminal/console but I want to supply input from the program — Saurabh, Mar 21 '16 at 06:11
Are you sure? For example, it implies that you can provide input manually despite `stdin=PIPE`. — jfs, Mar 21 '16 at 06:29
Yes, I can run 'some_script' manually in console/terminal and provide whatever input is required. All I want is to invoke it through a program automatically, so as the manual operation can be omitted. — Saurabh, Mar 21 '16 at 06:35
no. It is not what I said. Can you provide the input (type S on the keyboard) manually if you run the code from your question that contains `stdin=PIPE`? — jfs, Mar 21 '16 at 06:42
@J.F.Sebastian Can you reiterate your point? By default, stdin *is* the initial console, inherited from the python process. The entire point of `stdin=Pipe` is to give it a new pipe controlled by python. Unless one does some serious digging inside `/proc`, the subprocess cannot reconnect to the initial stdin. — MisterMiyagi, Mar 21 '16 at 13:52
@MisterMiyagi the code can read from terminal directly bypassing stdin. Here's [code example that prints to console directly](http://stackoverflow.com/a/20981435/4279) — jfs, Mar 21 '16 at 13:56
@J.F.Sebastian Seriously? Just assume that I placed ", etc" after mentioning `/proc`... — MisterMiyagi, Mar 21 '16 at 14:08
@MisterMiyagi what is your point? Reading from a tty directly is not exotic (it is common for applications that accept a user password, see [Q: Why not just use a pipe (popen())?](http://pexpect.readthedocs.org/en/stable/FAQ.html#whynotpipe)). `some_script` expecting an input from a tty may be one of the possible explanations of the observed behavior described in the question. — jfs, Mar 21 '16 at 14:57

score 1 · Answer 1 · answered Mar 18 '16 at 10:38

1

The entire point of Popen.communicate is that it will wait until the process terminates. If this is not desired behavior, you must explicitly interact with the process' stdin/stdout/stderr.

import subprocess
p = subprocess.Popen('some_script', stdin=subprocess.PIPE)
p.stdin.write('S\n')
p.kill()

answered Mar 18 '16 at 10:38

MisterMiyagi

44,374
10
104
119

Still the process is not being killed – Saurabh Mar 21 '16 at 03:05
1

@Saurabh `Popen.kill()` will send `SIGKILL` to the process. If the process survives this, the script you are calling is seriously flawed. Can you provide more information on what it's supposed to do? Is any I/O involved that may possibly block? – MisterMiyagi Mar 21 '16 at 13:46
2

@MisterMiyagi it is not surprising that `.kill()` may leave running processes behind. [SIGKILL always works](http://unix.stackexchange.com/q/5642/1321) but it doesn't kill the whole process tree e.g., see [How to terminate a python subprocess launched with shell=True](http://stackoverflow.com/q/4789837/4279) – jfs Mar 21 '16 at 20:46
@MisterMiyagi The shell script submits NASTRAN jobs in the cluster queue for running. For submitting the job I just have to hit 'S'.But from the python program, after 'S' is entered the shell script goes in a deadlock and any subsequent commands are left unexecuted. – Saurabh Mar 22 '16 at 03:52

Subprocess Deadlock

1 Answers1