13

I have multiple child "forked" by the same parent and I try to construct pipe connection between all these child processes like a linked list structure. Child 1 sends data to child2, child 2 to child 3.... child N to child 1. Is there any proper way to do it?

In addition, if I create and communicate between processes how I force the parent to "wait" all the process to finish their job since wait() or waitpid() waits for first finished process but I need to wait them all. It is the another question that arises.

Thanks...

osgx
  • 90,338
  • 53
  • 357
  • 513
erogol
  • 13,156
  • 33
  • 101
  • 155

3 Answers3

24

This is essentially what a shell does if build a redirection chain, i.e. something like

ls | grep foo | sort | uniq

There are some excellent introducionary texts on Unix programming, in which a simple shell is implemented through the book. And one of the tasks of a shell is redirection. One of these books is "Linux Application Programming" by Michael K. Johnson and Erik W. Troan.

The book's homepage: http://ladweb.net/

To build a chain of redirections for N processes you need N-1 pipes. For each redirection you create a pipe using the pipe(int fds[2]) system call. After fork()ing, but before execving use dup2(int from, int to) to "connect" a pipe's end to the standard input (0) or standard output of each process. Here's a overly simplified code, without error checking:

int pipe_A[2];
int pipe_B[2];

pipe(pipe_A);
pipe(pipe_B);

pid_t pid_A, pid_B, pid_C;

if( !(pid_A = fork()) ) {
    dup2(pipe_A[1], 1); /* redirect standard output to pipe_A write end */
    execv(...);
}

if( !(pid_B = fork()) ) {
    dup2(pipe_A[0], 0); /* redirect standard input to pipe_A read end */
    dup2(pipe_B[1], 1); /* redirect standard output to pipe_B write end */
    execv(...);
}

if( !(pid_C = fork()) ) {
    dup2(pipe_B[0], 0); /* redirect standard input to pipe_B read end */
    execv(...);
}

Take note that the pipe's array indices have been choosen in a way that they reflect the standard input/output file descriptors if they're used for stdio redirection. This choice was not arbitrary.

Of course you can connect pipes to any file descriptors (e.g. there are some applications, which expect their parent to open, say fd 3 and 4, connected to pipes) and most shells directly support this, too (for example 1>&3 will redirect stdout into fd 3). However the array indices for pipe(int fds[2]) are 0 and 1 of course. I'm just telling this, because I had some cargo-cult-programming students, which mindlessly took the target fds also for the pipe syscall array.

To wait for all the children to finish use waitpid(-1, NULL, 0) – I think that's the -1 my pre-answerer meant, which means: Wait for all child processes to finish. The other option was calling wait() in a loop which will return the pid of the just terminated child. If called again and there's still a child running, it will block again. If there's no child left, it will return -1; I prefer the waitpid solution.

datenwolf
  • 159,371
  • 13
  • 185
  • 298
  • quick question, when i use the above code my program hangs on the 2nd execv, i THINK cos it is waiting for a EOF for the stdin. how are i meant to close the stdin? – TrewTzu Sep 04 '11 at 05:22
  • 1
    @TrewTzu: You must not close stdin after it has been replaced with dup2(pipe_A[0], 0), since it would close the pipe then. I also don't see why execv should block. However execv will not return, it replaces the process with the called program. That's why the process is forked beforehand. On which OS are you trying this? Just curios. – datenwolf Sep 04 '11 at 09:13
  • 1
    This is a bit late, but looking at this code, shouldn't the subsequent if-statements be "else if" instead? Otherwise both the child and the parent of the first fork will run the second. – Jacob Sharf Apr 23 '13 at 18:05
  • 1
    @JacobSharf: No, because `exec…` replaces the process image. Everthing after the `exec…` within the if statement is not executed. – datenwolf Apr 23 '13 at 19:05
  • The link to the book is broken, you can get access to it through https://archive.org/details/linuxapplication0000john/page/n583/mode/2up – cassepipe Jan 04 '22 at 11:03
5

Yes, this is fairly easy, you just need to create all the pipes in the parent, and remember to close the pipes / ends of pipes in the child(ren) that you don't need them in.

Leaving FDs of the pipes open in children which aren't using them is a FAIL as it can make others wait forever for the end of the pipe. All writers must close before the reader gets an EOF.

MarkR
  • 62,604
  • 14
  • 116
  • 151
2

Create all the pipes first, then spawn all the children with the appropriate pipe ends in FDs 0 and 1.

As for waiting, just keep waiting until it returns -1.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358