Why subprocess does not terminate with Python 2.7 only?

Question

I have a code that spawns a process with subprocess.Popen:

from subprocess import check_call, CalledProcessError, Popen, PIPE
cmd="while true; do echo 123; done | grep -m1 123"
proc = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()

When I run it with Python 3.9.2, it terminates immediately as expected. However when I run it with Python 2.7 it hangs. It seems that Python 2.7 waits for shell infinite loop to terminate but it will not terminate ever. Can I make this code terminate under Python 2.7 as well?

It's not immediately obvious to me that it should return. Popen started a shell process. That process is still running, even though `grep` has closed the `stdout` handle. — Tim Roberts, Mar 03 '21 at 18:46
When `grep` terminates it should also terminate `while true` loop. This can be observed in bash interactive shell. This is what happens in Python 3.9.2 as I see, but not in Python 2.7. — ks1322, Mar 03 '21 at 18:55

score 0 · Accepted Answer · answered Mar 04 '21 at 17:24

Found related question Python subprocess.Popen blocks with shell and pipe for Python 2.7. The problem is that in Python 2.7 SIGPIPE signal is ignored by default and therefore while true loop ignores the fact that grep terminates on first match. There is unresolved Python issue for this https://bugs.python.org/issue1652. On Python 3 there is an extra parameter restore_signals=True for subprocess.Popen and therefore the code above works as is. For Python 2.7 it can be modified to restore SIGPIPE signal this way:

from subprocess import check_call, CalledProcessError, Popen, PIPE
import signal

def restore_signals():
    signals = ('SIGPIPE', 'SIGXFZ', 'SIGXFSZ')
    for sig in signals:
        if hasattr(signal, sig):
            signal.signal(getattr(signal, sig), signal.SIG_DFL)

cmd="while true; do echo 123; done | grep -m1 123"
proc = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE, preexec_fn=restore_signals)
out, err = proc.communicate()

Why subprocess does not terminate with Python 2.7 only?

1 Answers1