1

I'm trying to understand this this answer. Especially, how does data flow across processes?

I assume that input stream flows through a child's input through output connected with pipe, then is collected by a parent through pipe output. However, before the new turn of the loop and executing the command in a child, what does happen to the read data?

Is data buffered inside the parent process (somehow? I'd like to know what's responsible for that), and later this buffer is copied and passed to a child after fork(), then the child passes the saved input to the exec()?

EDIT - additional aspect:

(In the example I've presented, the new pipe is created with every turn of the loop and filled with new descriptors, when pipe() is called. I've created a working modification, where the table of descriptors is created outside the loop and repopulated. How does it affect the previously read data from previously executed command? Where it is stored, when the new pipe is created and new file descriptors are stored in the reused table?)

wannalearn
  • 11
  • 2

1 Answers1

1

There is a buffer associated with the pipe.

If the buffer isn't full, the write will return and the program will continue.

But it is possible for the buffer to fill up. If a write can't complete, it will either result in a partial write (with the amount written provided) or block until a the write can be completed (i.e. when the something ends up reading from the other end).

Readers trying to read from an empty pipe usually block until data becomes available. If the descriptor was made non-blocking, the read will return error EWOULDBLOCK or EAGAIN instead.

select and similar mechanisms can be used to detect when it's safe to perform a read or write without blocking.

Each end of the pipe can be closed independently. The end is only closed once all copies of the file descriptor is closed. This includes inherited copies. A common scenario is for a child to inherit a pipe from its parent. The parent will close one of its descriptors, and the child will close the other. This leaves one process the ability to write to the pipe, and the other the ability to read from it. (Bi-directional communication requires two pipes or a socket.)

What happens if the reader exits? By default, writing to a closed pipe results in the SIGPIPE signal. And by default, this kills the writing process. This is often very useful. For example, consider the situation where head is used on a very long stream. But it's not always useful. Sometimes, you want to detect and handle the situation. Fortunately, this is fully configurable by handling the signal or ignoring it, using the usual mechanisms. If ignored, the write operation will fail with error code EPIPE.

There will be variance in behaviour between systems. For example, Windows has no signals*, and select doesn't work on pipes. (It does work on sockets, though.) But it also provides alternate means of performing asynchronous I/O.

ikegami
  • 367,544
  • 15
  • 269
  • 518
  • In the example I've presented, the new pipe is created with every turn of the loop and filled with new descriptors, when `pipe()` is called. I've created a working modification, where the table of descriptors is created outside the loop and repopulated. How does it affect the previously read data from previously executed command? – wannalearn Dec 07 '21 at 08:56
  • I have a thought on my follow-up question: the new pipe is created in the same table, but the old pipe isn't destroyed - that's where the data is waiting, in the buffer of the previous pipe. The child in the new turn of the loop inherits the descriptors table from the parent - in this descriptor table, there is duplicated previous pipe's reading descriptors. The child duplicates the writing descriptor, which hasn't been altered by the parent and redirects the output to the newly created pipe. – wannalearn Dec 07 '21 at 09:08
  • (Exceeded letters limit) However, in the comments to the cited question, close(pipe[0]) has been suggested. How does it affect reading of the data from the previous pipe in the next turn of the loop? – wannalearn Dec 07 '21 at 09:09
  • OK, the descriptor in descriptors table still isn't closed, therefore we can read from the previous pipe. – wannalearn Dec 07 '21 at 09:30