Setting up a pipeline using subprocess.Popen: Why close stdout but not stdin?

Question

The Python 3 documentation for subprocess.popen (1) presents the following sample code for pipelines:

from subprocess import Popen, PIPE
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

The p1.stdout.close() call after starting the p2 is important in order for p1 to receive a SIGPIPE if p2 exits before p1.

Why is this necessary? Previous questions (Replacing Shell Pipeline, Under what condition does a Python subprocess get a SIGPIPE?, Explain example from python subprocess module) have answers stating that p1.stdout has multiple readers, which must all be closed to prevent p1's output pipe from closing.

What is the relation between p2.stdin and p1.stdout, and do I have to close p2.stdin?

score 0 · Accepted Answer · answered Aug 28 '18 at 08:05

No, you do not have to close p2.stdin. In fact you can't, because it's None.

This is because stdin=p1.stdout, while a Python stream object is only created if stdin=subprocess.PIPE. See subprocess Popen.stdin docs)

I wrote a test program (Python 3) which terminates with BrokenPipeError if CLOSE_P1 = True, but spins forever?? if CLOSE_P1 = False:

from subprocess import Popen, PIPE


CLOSE_P1 = True


p1 = Popen("ffmpeg "
           "-f rawvideo -pixel_format rgb24 -video_size 1x1 -i - "
           "-c copy -f rawvideo -".split(), stdin=PIPE, stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
assert p2.stdin is None

if CLOSE_P1:
    p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
p2.terminate()
while 1:
    p1.stdin.write(b'fsdafhdsf9jd9s8ha0')

Setting up a pipeline using subprocess.Popen: Why close stdout but not stdin?

1 Answers1