multi-pipe C program does not work

Question

I decided to write a simple, hard coded c program to better understand how pipes work. The program has 3 commands: find . -name '.' | sed 's/.*\.// | sort

And it works with 1 pipe (if i use only 2 commands) but it fails with 2 pipes(sort just does not get the information).

I think the error is in close or waitpid, but I have tried everything(i could think of) and it still does not work. What am I missing ? information i used:

Is it possible to have pipe between two child processes created by same parent (LINUX, POSIX)

http://www.quora.com/Unix/How-can-I-write-a-code-for-PIPE-in-C-shell-script-python <--Sams example

Implementation of multiple pipes in C

EDIT: the commands are written with no mistakes. The problem is definitely not in them (since they work if I only use 1 pipe) My code:

#include <errno.h>
#include <stdio.h>
#include <unistd.h>  

int main(int argc, char* argv[]) {


    pid_t test, test2, test3;
    int old_pipe[2];
    int new_pipe[2];


    //test command
    //char *argv1[] = {"ls", "-1", "-h", NULL};
    char *argv1[] = {"find", ".", "-name", "*.*",NULL};
    char *argv2[] = {"sed", "s/.*\\.//", NULL};
    char *argv3[] = {"sort", NULL};

    //creates a pipe
    pipe(old_pipe);
    pipe(new_pipe);

    test = fork();
    if(test == 0){
        dup2(old_pipe[1], 1);
        close(old_pipe[0]);

        execvp(argv1[0], argv1);
            perror("exec");
            return 1;
    }

    test2 = fork();
    if(test2 == 0){
        dup2(old_pipe[0], 0);
        close(old_pipe[1]);
        dup2(new_pipe[1], 1);
        close(new_pipe[0]);

        execvp(argv2[0], argv2);
            perror("exec");
            return 1;
    }


    test3 = fork();
    if(test3 == 0){
        dup2(new_pipe[0], 0);
        close(new_pipe[1]);

        execvp(argv3[0], argv3);
        perror("exec");
        return 1;
    }

    close(old_pipe[0]);
    close(old_pipe[1]);

    close(new_pipe[0]);
    close(new_pipe[1]); 

    waitpid(test);
    waitpid(test2);
    waitpid(test3);

    return 0;
}

My first suggestion when writing your own `system()` function as well as implementing the `|` pipe concept is to implement with functions so it is easier to reason about your code, rather than 3 inline forks that become unclear. Move the logic out of main and abstract it. — codenheim, Oct 25 '14 at 17:19
Yes you are 100% correct about splitting my logic. But I just wanted to see how pipes work, before trying to implement them the right way and I got stuck at this simple example(that I think SHOULD work). — Stralos, Oct 25 '14 at 17:25
Close files. The child that execs `find` has both ends of new_pipe open! After the dup2, you should be closing four file descriptors. — William Pursell, Oct 25 '14 at 18:01

ensc · Accepted Answer · 2014-10-25T18:04:24.307

1

your 3rd exec (starting "sort") does not close old_pipe[1] and keeps it open. sed will not see the EOF and stays alive. You should call pipe(2) very late.

I suggest to look into /proc/<pid>/fd resp. use lsof to see which filedescriptors are open. E.g. after

    dup2(old_pipe[1], 1);
    close(old_pipe[0]);

you should close old_pipe[1] when it is not 1.

Some more explanations (as asked in comment): You program has

pipe(old_pipe);
pipe(new_pipe);

test = fork();
   /* --> 'find'; writes into old_pipe[1] */
test2 = fork();
   /* --> 'sed'; reads from old_pipe[0], writes into new_pipe[1] */
test3 = fork();
   /* --> 'sort'; reads from new_pipe[0] */

The test2 program does not exit as long as its input (old_pipe[0]) is open. Because this pipe is alive for test3 (which waits for test2 to finish), it will deadlock.

Closing fds in child branches does not close them in the parent process.

edited Oct 25 '14 at 18:04

answered Oct 25 '14 at 17:41

ensc

6,704
14
22

It works! can you also explain why do I have to close old_pipe[0] even if I closed it before ? So if I would have 5 pipes active at one moment, I would need to close each one of them at every child ? – Stralos Oct 25 '14 at 17:52
What you should do is to close everything that you will not use. So if you create a pipe and then fork, you should close in both processes everything not useful. The writing process should immediately close the reading part, and the reader process should immediately close the writing part of the pipe. Remember that descriptors are inherited across forks, so if you open something before a fork, after, the two processes have a proper copy of the initial descriptor. – Jean-Baptiste Yunès Oct 27 '14 at 07:03

multi-pipe C program does not work

1 Answers1