3

I am reading the the Python documentation on the Popen class in the subprocess module section and I came across the following code:

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

The documentation also states that

"The p1.stdout.close() call after starting the p2 is important in order for p1 to receive a SIGPIPE if p2 exits before p1.

Why must the p1.stdout be closed before we can receive a SIGPIPE and how does p1 knows that p2 exits before p1 if we already closed it?

user40990
  • 69
  • 5

1 Answers1

6

SIGPIPE is a signal that would be sent if dmesg tried to write to a closed pipe. Here, dmesg ends up with two targets to write to, your Python process and the grep process.

That's because subprocess clones file handles (using the os.dup2() function). Configuring p2 to use p1.stdout triggers a os.dup2() call that asks the OS to duplicate the pipe filehandle; the duplicate is used to connect dmesg to grep.

With two open file handles for dmesg stdout, dmesg is never given a SIGPIPE signal if only one of them closes early, so grep closing would never be detected. dmesg would needlessly continue to produce output.

So by closing p1.stdout immediately, you ensure that the only remaining filehandle reading from dmesg stdout is the grep process, and if that process were to exit, dmesg receives a SIGPIPE.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • @MartijnPieters Why is `p1` still writing to stdout though? If python truly calls `dup2(p1.stdout, PIPE)`, then that means that `p1` should not be making any writes to stdout anywhere. When the pipe is closed from p2's end, that will send a proper SIGPIPE. Or does Python not really call dup2, but just creates another file descriptor that it writes to when stdout is written to? – darksky May 23 '17 at 17:21
  • @darksky: Python calls `dup2(p1.stdout.fileno(), 0) here, so replacing stdin of the child process with the pipe file number. Why does this mean that `p1 should not be making any writes to stdout? `p1.stdout` is **one end of a pipe**, and `p1` is writing to the other end. `p1.stdout` is just the Python process reference to the receiving end of the pipe. – Martijn Pieters May 23 '17 at 17:57
  • @darksky: Python calls `dup2(p1.stdout.fileno(), 0)` here, so replacing stdin of the child process with the pipe file number. Why does this mean that `p1 should not be making any writes to stdout? `p1.stdout` is **one end of a pipe**, and `p1` is writing to the other end. `p1.stdout` is just the Python process reference to the receiving end of the pipe. – Martijn Pieters May 23 '17 at 17:57