0

I have a process P which has to keep track of a list of processes P1 to Pn.

The program is initialized with P1, the head of the list, already created. On a specific event (not in the scope of the question), P has to write trough a pipe to the current last process Pi in the list and this process, upon receiving the message, has to fork. The newly created Pi+1 has to be added to a linked list managed by P as the new last process.

The way I'm trying to implement this is the following:

  • P initializes an empty linked-list List, creates a pipe childs_to_p to receive messages from all descendants, creates a pipe fd, forks P1 and appends it to List.
  • P1 starts listening on fd.
  • If P wants to add another process, it creates a new pipe fd2, and through fd it communicates to P1 (the last process in List for this first iteration) the reading end of fd2 (the reading end that P2 will have to listen from).
  • P1 reads from fd, forks and tells the newly created P2 the reading end of fd2. Then continues to listen for instructions on fd.

  • P2 sends its pid to P through the pipe childs_to_p.

  • P adds P2'pid to List.
  • Repeat n times.

Keep in mind that I'm new to C language, but here's what I implemented so far.

This is the initialization of P:

void setupProcessManager(int we, int re, int msg_size) {
     write_to_m = we;             // Setup of pipe to external
     read_from_m = re;            // process that gives instructions
     message_size = msg_size;     // to P.

     initList(&l);                // Linked list initialization

     open_childs_to_pm_pipe();    // Open pipe to read from processes in list

     addp1();                     // Initialize P1

     pthread_t t;               
     pthread_create(&t, NULL, PMListen, NULL);   // P starts listening from external process


     pthread_join(t,NULL);        // Wait for thread termination

     close(write_to_m);           // Close pipe to external process
     close(read_from_m);          // Close pipe from external process
}

The function to initialize P1:

void addp1() {
     int to_p1[2];           
     pipe(to_p1);              // Create new pipe to P1
     pid_t pid = fork();       // Fork P1
     if(pid == 0) {
         PListen(to_p1[0]);    // P1 starts listening
         exit(0);
     } else if(pid>0) {
         insertEnd(&l, pid, to_p1[1], to_p1[0]);  // P appends P1 to list 
     }
 }

The function performed by P (consider only the 'A' case in the switch):

void *PMListen() {
     int message;
     int listen = 1;
     while(listen) {
         read(read_from_m,&message,message_size);
         switch(message) {

             // OTHER CASES

             case 'A':   
                 int new_pipe[2];   // P creates a new pipe
                 pipe(new_pipe);
                 write(getLastProcessWritingEnd(&l), &new_pipe[0], message_size);  // P writes reading end of new pipe to P1
                 pid_t pid;
                 read(childs_to_pm[0], &pid,sizeof(pid_t)); // P reads P2 pid from childs_to_p
                 insertEnd(&l, pid, new_pipe[1], new_pipe[0]); // P appends P2 to list
                 break;

         }
     }

 }

And finally the task performed by the processes in the list:

void PListen(int read_from_here) {
     int message;
     int listen = 1;
     while(read(read_from_here,&message,message_size) != 0) {
         switch(message) {
             default:                  // Pi reads integer (reading end of pipe P-Pi+1)
                 pid_t pid = fork();   // Fork Pi+1
                 if(pid == 0) {
                     pid_t mypid = getpid();
                     write(childs_to_pm[1], &mypid, sizeof(int)); // Pi+1 writes its pid to P through childs_to_p
                     PListen(message); // Pi+1 starts listening on its pipe from P
                     exit(0);
                 }
                 break;
         }
     }

 }

Everything works as intended. P1 is created and added to the linked list. When P tries to add P2 it does so but then the function PListen keeps looping, forking forever.

I know the code is not well written, but as I mentioned I'm not familiar with C. I need help with the algorithm rather than with the beauty of the code.

ducky-typed
  • 53
  • 1
  • 9
  • 1
    AFAIK, you can't send file descriptors over pipes. It's possible to [pass file descriptors over sockets](https://stackoverflow.com/a/28005250/15168), but not over pipes. So, I think there are some problems with this design. Processes sharing a pipe must have a common ancestor that created the pipe before the processes were created (or one of them must be the ancestor that created the pipe). – Jonathan Leffler Apr 08 '20 at 20:04
  • I think you're going to need to create an MCVE ([Minimal, Complete, Verifiable Example](https://stackoverflow.com/help/mcve)) (or MRE or whatever name SO now uses) or an SSCCE ([Short, Self-Contained, Correct Example](http://sscce.org/)). It's rather difficult to see how everything works, especially since there are many global variables for which we cannot see the definitions. – Jonathan Leffler Apr 08 '20 at 20:12
  • `PMListen()` is supposed to be a thread function, but it has the incorrect signature. It probably isn't fatal, but you shouldn't take chances. Mixing threads with forking is usually not a good idea; you need to be cautious, at any rate. – Jonathan Leffler Apr 08 '20 at 20:14
  • Good luck if you can only use unnamed pipes, as a transient comment intimated. I'll look to see how you resolved the problems, then. There may be possibilities on Linux that I'm not familiar with, but I think you've got a rocky road ahead of you. – Jonathan Leffler Apr 08 '20 at 20:15
  • @JonathanLeffler The fact is that for this program I should use only unnamed pipes, so that's why I decided to go towards this implementation and design. In my case P IS indeed the ancestor that creates the pipe – ducky-typed Apr 08 '20 at 20:20
  • Yes, but the pipes must all be created before `P1` is created. And that's not what you're trying to do, AFAICT. I don't see code related to waiting for dead processes; on the whole, that's probably not a bad thing as `P` cannot wait for the processes that `P1` creates. Only the parent of a process can wait for it — ignoring the system process that inherits orphaned processes and waits for them to die. So only `P1` can wait for `P2` et al. – Jonathan Leffler Apr 08 '20 at 20:23
  • @JonathanLeffler I already cut out a lot of code trying to address the problem with more focus, but I admit it's kinda messy and it's a rather specific problem I'm facing. May I ask how would you roughly design a task such as this one? – ducky-typed Apr 08 '20 at 20:25
  • @JonathanLeffler Yes I realize that, I was going one step at a time, first trying to understand how to create new processes and only then how to manage their termination. I know about wait() and zombie processes, I just didn't implement it yet. – ducky-typed Apr 08 '20 at 20:27
  • I don't know, other than "not like this". I don't understand what the system is trying to do sufficiently well. I'd not use any threads. It's particularly gratuitous when you have the function doing `pthread_create()` and immediately `pthread_join()` for a single thread. I'd probably have the main process, `P`, forking all the children, creating pipes, and closing pipes (carefully) as it goes. I'm not clear why the children need to communicate with each other. It's mysterious. – Jonathan Leffler Apr 08 '20 at 20:28
  • @JonathanLeffler P, as the task was given to me, can only fork P1, then there must be a chain of forking from P1 to P2, P2 to P3 and so on. And these do not have to communicate to each other, but have each an individual pipe from P. – ducky-typed Apr 08 '20 at 20:32
  • Then P has to create _n_ pipes before forking `P1`, and `P1` has to know how to manage the pipes. `P1` can't close them until the relevant children are forked, but those children must know which pipes to close and which to use, and so on. I'm still very unclear about what all these children are doing — how `P` is communicating with them, or how they're communicating with `P`. As I said, I'm not understanding what this system is supposed to achieve, which limits the amount of help I can offer. Unless there is a radical rewrite of the question, I'm ducking out of the conversation. I can't help. – Jonathan Leffler Apr 08 '20 at 20:37
  • @JonathanLeffler Just for the sake of answering the questions in your last comment: I can't create n pipes at the beginning because I don't know how many processes I'll have to create later, so the pipes have to be initialized on the fly. P should communicate with each descendant via a "private" pipe. And the only purpose of descendants processes is to wait for an instruction from P. It's a purely didactic project, to understand concepts of processes, it doesn't have any practical use. That said, I'm not going to blame you for ducking out, on the contrary, thanks for your comments! – ducky-typed Apr 08 '20 at 20:54
  • What you could do is: (1) P creates two pipes, one to write to children, one to read from children; (2) P forks the first child, P1; (3) P closes the read end of the 'to children' pipe and the write end of the 'from children' pipe; (4) P waits for events — e.g. reads a line from standard input, and springs into action when one arrives; (5) P1 closes the write end of the 'to children' pipe and the read end of the 'from children' pipe; (6) it waits for a message from P on the 'to children' pipe; (7) P writes a message on the 'to children' pipe; (8) P1 reads the message; _[…continued…]_ – Jonathan Leffler Apr 08 '20 at 21:07
  • _[…continuation…]_ (9) P1 forks a child P2; (10) P1 writes the PID of P2 on the 'from children' pipe and closes it, and also closes the 'to children' pipe (it won't use them again); (11) P reads the PID of P2 from the 'from children' pipe; (12) P goes back to waiting for another event (line of input); things continue from step (4) with a new child, the latest in the list, reacting to each event (message from P). I'm not sure if that meets your requirements, but it could work. Each generation of child process could exit after it closes the pipes (or it could exit in order to close the pipes). – Jonathan Leffler Apr 08 '20 at 21:10

1 Answers1

1

There is extensive discussion in the comments of why I think the design trying to pass file descriptors for newly created pipes from the initial process P to the Nth-generation child processes won't work as described. In short, once a child process P1 has been created, there is no way for its parent, P, to create new file descriptors (e.g. by a call to pipe()) that can be relayed to the child process.

I outlined an alternative design that could — and does – work:

  1. P creates two pipes, one to write to children, one to read from children;
  2. P forks the first child, P1;
  3. P closes the read end of the 'to children' pipe and the write end of the 'from children' pipe;
  4. P waits for events — e.g. reads a line from standard input, and springs into action when one arrives;
  5. P1 closes the write end of the 'to children' pipe and the read end of the 'from children' pipe;
  6. P1 waits for a message from P on the 'to children' pipe;
  7. P writes a message on the 'to children' pipe;
  8. P1 reads the message;
  9. P1 forks a child P2;
  10. P1 writes the PID of P2 on the 'from children' pipe and closes it, and also closes the 'to children' pipe (it won't use them again);
  11. P reads the PID of P2 from the 'from children' pipe;
  12. P goes back to waiting for another event (line of input)

Things continue from step (4) with a new child, the latest in the list, reacting to each event (message from P). Each generation of child process could exit after it closes the pipes (or it could exit in order to close the pipes).

The only two pipes are created by the initial process, and the relevant file descriptors are inherited by the next generation of child in turn.

Here is working code that implements this scheme. The original child, P1, can arrange for its standard input to be the read end of the 'to children' pipe and its standard output to be the write end of the 'from children' pipe. This allows the children to use getline() to read messages from the initial process, P. They actually write binary data on the pipe with fwrite(); it would be feasible to make it into string data, but it isn't necessary (and it would make the original parent process a little more complicated).

It uses some error reporting code which is available in my SOQ (Stack Overflow Questions) repository on GitHub as files stderr.c and stderr.h in the src/libsoq sub-directory. In particular, it sets the error options to report the PID of each process reporting (very helpful when dealing with multiple processes as here).

Source file pipe73.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "stderr.h"

static void be_parental(int to_kids[2], int fr_kids[2]);
static void be_childish(int to_kids[2], int fr_kids[2]);

int main(int argc, char **argv)
{
    if (argc > 0)
        err_setarg0(argv[0]);
    err_setlogopts(ERR_PID|ERR_MILLI);
    err_remark("Parent at work\n");

    int to_kids[2];
    int fr_kids[2];

    if (pipe(to_kids) < 0 || pipe(fr_kids) < 0)
        err_syserr("failed to create pipes: ");

    int pid = fork();
    if (pid < 0)
        err_syserr("failed to fork: ");
    else if (pid == 0)
        be_childish(to_kids, fr_kids);
    else
        be_parental(to_kids, fr_kids);

    /*NOTREACHED*/
    err_error("Oops!\n");
    return EXIT_FAILURE;
}

static void be_childish(int to_kids[2], int fr_kids[2])
{
    int child = 0;
    if (dup2(to_kids[0], STDIN_FILENO) < 0 ||
        dup2(fr_kids[1], STDOUT_FILENO) < 0)
        err_syserr("failed to duplicate pipes to standard input/output: ");

    close(to_kids[0]);
    close(to_kids[1]);
    close(fr_kids[0]);
    close(fr_kids[1]);

    err_remark("child P%d at work\n", ++child);

    char *buffer = 0;
    size_t buflen = 0;
    ssize_t msglen;
    while ((msglen = getline(&buffer, &buflen, stdin)) != -1)
    {
        err_remark("Read: [%.*s]\n", (int)msglen - 1, buffer);
        int pid = fork();
        if (pid < 0)
            err_syserr("failed to fork: ");
        else if (pid != 0)
        {
            size_t nbytes = fwrite(&pid, sizeof(char), sizeof(pid), stdout);
            if (nbytes != sizeof(pid))
            {
                if (nbytes == 0)
                    err_syserr("write to pipe failed: ");
                else
                    err_error("short write of %zu bytes instead of %zu expected\n", nbytes, sizeof(pid));
            }
            err_remark("forked PID %d\n", pid);
            err_remark("child P%d finished\n", child);
            exit(0);
        }
        else
            err_remark("child P%d at work\n", ++child);
    }

    free(buffer);
    err_remark("EOF\n");
    err_remark("child P%d finished\n", child);
    exit(0);
}

static void be_parental(int to_kids[2], int fr_kids[2])
{
    close(to_kids[0]);
    close(fr_kids[1]);
    char  *buffer = 0;
    size_t buflen = 0;
    ssize_t msglen;
    int pid;

    err_remark("parent at work\n");

    while ((msglen = getline(&buffer, &buflen, stdin)) != -1)
    {
        err_remark("message: %s", buffer);  /* buffer includes a newline - usually */
        ssize_t nbytes = write(to_kids[1], buffer, msglen);
        if (nbytes < 0)
            err_syserr("write to pipe failed: ");
        else if (nbytes != msglen)
            err_error("short write of %zu bytes instead of %zu expected\n", (size_t) nbytes, (size_t)msglen);
        nbytes = read(fr_kids[0], &pid, sizeof(pid));
        if (nbytes < 0)
            err_syserr("read from pipe failed: ");
        else if ((size_t)nbytes != sizeof(pid))
            err_error("short read of %zu bytes instead of %zu expected\n", (size_t) nbytes, sizeof(pid));
        err_remark("PID %d started\n", pid);
    }

    err_remark("EOF\n");

    close(to_kids[1]);
    close(fr_kids[0]);

    free(buffer);
    err_remark("Finished\n");

    exit(0);
}

Example run of pipe73

The typed inputs are 'abracadabra', 'hocus-pocus', 'zippedy-doo-dah', 'absolute zero' and 'boiling point'; after that, I indicated EOF (typed Control-D).

$ pipe73
pipe73: 2020-04-08 16:17:59.531 - pid=14441: Parent at work
pipe73: 2020-04-08 16:17:59.532 - pid=14441: parent at work
pipe73: 2020-04-08 16:17:59.532 - pid=14442: child P1 at work
abracadabra
pipe73: 2020-04-08 16:18:03.095 - pid=14441: message: abracadabra
pipe73: 2020-04-08 16:18:03.095 - pid=14442: Read: [abracadabra]
pipe73: 2020-04-08 16:18:03.096 - pid=14441: PID 14443 started
pipe73: 2020-04-08 16:18:03.096 - pid=14442: forked PID 14443
pipe73: 2020-04-08 16:18:03.096 - pid=14443: child P2 at work
pipe73: 2020-04-08 16:18:03.097 - pid=14442: child P1 finished
hocus-pocus
pipe73: 2020-04-08 16:18:08.989 - pid=14441: message: hocus-pocus
pipe73: 2020-04-08 16:18:08.989 - pid=14443: Read: [hocus-pocus]
pipe73: 2020-04-08 16:18:08.990 - pid=14441: PID 14444 started
pipe73: 2020-04-08 16:18:08.990 - pid=14443: forked PID 14444
pipe73: 2020-04-08 16:18:08.990 - pid=14444: child P3 at work
pipe73: 2020-04-08 16:18:08.990 - pid=14443: child P2 finished
zippedy-doo-dah
pipe73: 2020-04-08 16:18:14.711 - pid=14441: message: zippedy-doo-dah
pipe73: 2020-04-08 16:18:14.711 - pid=14444: Read: [zippedy-doo-dah]
pipe73: 2020-04-08 16:18:14.712 - pid=14441: PID 14445 started
pipe73: 2020-04-08 16:18:14.711 - pid=14444: forked PID 14445
pipe73: 2020-04-08 16:18:14.712 - pid=14445: child P4 at work
pipe73: 2020-04-08 16:18:14.712 - pid=14444: child P3 finished
absolute zero
pipe73: 2020-04-08 16:18:19.169 - pid=14441: message: absolute zero
pipe73: 2020-04-08 16:18:19.169 - pid=14445: Read: [absolute zero]
pipe73: 2020-04-08 16:18:19.170 - pid=14441: PID 14446 started
pipe73: 2020-04-08 16:18:19.170 - pid=14445: forked PID 14446
pipe73: 2020-04-08 16:18:19.170 - pid=14445: child P4 finished
pipe73: 2020-04-08 16:18:19.170 - pid=14446: child P5 at work
boiling point
pipe73: 2020-04-08 16:18:26.772 - pid=14441: message: boiling point
pipe73: 2020-04-08 16:18:26.772 - pid=14446: Read: [boiling point]
pipe73: 2020-04-08 16:18:26.773 - pid=14441: PID 14447 started
pipe73: 2020-04-08 16:18:26.773 - pid=14447: child P6 at work
pipe73: 2020-04-08 16:18:26.773 - pid=14446: forked PID 14447
pipe73: 2020-04-08 16:18:26.774 - pid=14446: child P5 finished
pipe73: 2020-04-08 16:18:29.374 - pid=14441: EOF
pipe73: 2020-04-08 16:18:29.374 - pid=14441: Finished
pipe73: 2020-04-08 16:18:29.374 - pid=14447: EOF
pipe73: 2020-04-08 16:18:29.375 - pid=14447: child P6 finished
$

That seems to cover the gist of what's required. The Nth-generation child relays to the original parent process the PID of the child it just launched, and the (N+1)th-generation is the one that responds to the next request. In this code, the parent doesn't keep a list of the children it is told about, but it does report on each of those children as it is told about it (saving the information would not be hard). Once an Nth-generation child has reported to P, it terminates. It could be revised to do other work as well.

Jonathan Leffler
  • 730,956
  • 141
  • 904
  • 1,278
  • That is a perfect solution for my task. I'll have to revise it just a little bit to meet the exact requirements of the project, but it does exactly what I asked. Thank you for your commitment in finding an answer, even though the task wasn't neither easy nor clearly explained. Kudos yo you! – ducky-typed Apr 09 '20 at 07:44