C -> Shell - Block on a Write Until Read

Question

I want to send a character from a c program to a shell program. I am using a named pipe to send the letter 'a' whenever it is requsted. I should only have to open the pipe once. Here's an example:

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
int main(){
    int fd;
    mkfifo("/tmp/test", 0666);
    fd = open("/tmp/test", O_WRONLY);
    printf("Opened\n");
    char * a = "a";
    while(1){
            printf("Writing to pipe...\n");
            write(fd,a,1);
            sleep(1);
    }
}

And the shell executes this command as many times as it wants...

head -c 1 /tmp/test

The issue is after one head, the c will endlessly stream into the pipe, even if nobody's there.

I noticed that open() blocks until someone is on the other end. How to I tell write() to block until somebody is reading?

I would rather have this feature on write() than read(), as I think there's lots of overhead on opening the file for each request.

Thanks!

UPDATE

This is how I'm handling it in Java, it waits until I have somebody listening on this pipe before it continues on. Maybe just because it's a higher level language.

public static void writeToPipe(int val, String pipename){
    try{
        pipe_out = new PrintStream("/tmp/" + pipename);
    }catch(Exception e){
        System.out.println("Could not open a pipe for output!");
        e.printStackTrace();
    }
    try{
        pipe_out.println(val);
    }catch(Exception e){
        System.out.println("Could not write to pipe!");
        e.printStackTrace();
    }

    try{
        pipe_out.close();
    }catch(Exception e){
        System.out.println("Could not close the output pipe!");
        e.printStackTrace();
    }
}

UPDATE #2 - THIS IS THE SOLUTION

Here is my code based on David's idea, it's rough, but it works. I'm not checking if the named pipe exists and just supressing it from quitting.

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <sys/wait.h>
#include <stdlib.h>
int main(int argc, char **argv){
    mkfifo("/tmp/test", 0666);
    while(1){
      int fd, status;
            if ((fd = open ("/tmp/test", O_WRONLY)) == -1) {
         perror ("open failed");
         return 1;
            }
     printf("Opened Pipe\n");
     char a = 'a';
            int f  = fork();
            if(f == -1){
                    perror("fork");
                    exit(1);
            }else if(f == 0){
                    //This is by the child process
                    if (write (fd, &a, 1) == -1) {
                            close(fd);
                    perror ("open failed");
                            return 1;
         }

            }else{
                    //This is the parent process
                    int w = waitpid(f, &status, WUNTRACED | WCONTINUED);
                    if (w == -1){
                            perror("waitpid");
                            exit(EXIT_FAILURE);
                    }
            }
    }
}

It's probably a buffering thing. If you wrote enough data to the pipe, the write would block until someone started reading from it (or perhaps the write would return EAGAIN or similar). — Neil, Jul 07 '17 at 15:40

William Pursell · Answer 1 · 2017-07-07T16:38:30.780

0

Since you don't bother to check the return of write, you don't even notice that it is failing. (That is, it's not endlessly streaming into the pipe; it's just lying to you and printing "Writing to pipe..." while failing to write to the pipe.)

The only way to block on the write is to fill the pipe. But that's not your problem. The problem is that the pipe has been terminated. If you want to keep that pipe open, you'll need to have some process still alive reading from it. For example, you could do sleep 500 < /tmp/test in some other shell. Or just open the fifo for reading in the program doing the writing. (eg, add the line open("/tmp/fifo", O_RDONLY);)

The more curious issue is; why isn't your program terminating from the SIGPIPE?

edited Jul 07 '17 at 16:38

answered Jul 07 '17 at 16:21

William Pursell

204,365
48
270
300

Okay, how does it 'failing' help me code a solution that keeps it from polling the write()? I could just sleep and try again, but that defeats the purpose of blocking. – jrcichra Jul 07 '17 at 16:29
@Demodude123 Edited with more detail and some suggestions on making the write block. – William Pursell Jul 07 '17 at 16:47
it's a good idea, but either of those ways causes the C program to keep printing to the pipe. It's not upon request. The sloppy way to fix all this is have the open() inside the loop, but I'm trying to avoid recreating the file descriptor each call to this C code. – jrcichra Jul 07 '17 at 17:30
It only prints until the pipe buffer is full. If you want no data to be generated until some consumer is ready for it, then pipes are the wrong form of IPC. But, you could possibly change the size of the pipe buffer to make this suit your purpose: https://stackoverflow.com/questions/4739348/is-it-possible-to-change-the-size-of-a-named-pipe-on-linux – William Pursell Jul 07 '17 at 17:40
Trying to run this on kernel 2.6.32, doesn't support that, but awesome find. – jrcichra Jul 07 '17 at 17:52

David C. Rankin · Accepted Answer · 2017-07-07T19:28:06.433

You can do what you are attempting, but understand you must limit your read to one-char on the shell side since there will be no '\n' written to the pipe. Also, you may write many more times than the shell reads. For example, you can add validations as Mr. Pursell suggests to insure your C-program is functioning and blocking on write with something similar to:

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>

int main (int argc, char **argv) {

    int fd;
    errno = 0;

    if (mkfifo (argc > 1 ? argv[1] : "/tmp/test", 0666)) {
        perror ("mkfifo failed");
        return 1;
    }
    if ((fd = open ("/tmp/test", O_WRONLY)) == -1) {
        perror ("open failed");
        return 1;    
    }

    printf ("Opened\n");
    char a = 'a';

    while (1) {
        printf ("Writing to pipe...\n");
        if (write (fd, &a, 1) == -1) {
            perror ("open failed");
            return 1;
        }
    }

    return 0;
}

You can test with a simple:

$ declare -i c=0; while test "$c" -lt 10 && read -n 1 ch; do 
    echo "read: $ch"
    ((c++))
done </tmp/test

Example Shell Output

read: a
read: a
read: a
read: a
read: a
read: a
read: a
read: a
read: a
read: a

You will write until the fifo buffer is full resulting in more Writing to pipe... than you have read: a.

Rough Example with fork

Continuing from the comments here is a rough example of using fork to spawn child processes to insure your C-program is always blocking on write for the shell. This example is limited to 3 repetitions, but you could just use while (1) for a continual cycle. I also added a quick counter for the write (just for my curiosity) e.g.

#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

int crt_fifo_write (char *fname);
int file_exists (char *f);

int main (int argc, char **argv) {

    int n = 0;
    errno = 0;

    while (n < 3) {         /* limit example to 3 child processes */

        pid_t cpid, w;
        int status;

        cpid = fork();
        if (cpid == -1) {
            perror("fork");
            exit(EXIT_FAILURE);
        }

        if (cpid == 0) {    /* Code executed by child */
            if (!crt_fifo_write (argc > 1 ? argv[1] : "/tmp/test"))
                fprintf (stderr, "crt_fifo_write() failed.\n");
        } 
        else {              /* Code executed by parent */
            do {
                w = waitpid (cpid, &status, WUNTRACED | WCONTINUED);
                if (w == -1) {
                    perror("waitpid");
                    exit(EXIT_FAILURE);
                }
                if (WIFSIGNALED(status)) /* signal on close of read end */
                    printf("shell read complete. %s\n", 
                            n < 2 ? "restarting" : "done");
            } while (!WIFEXITED(status) && !WIFSIGNALED(status));
        }
        n++;
    }

    return 0;
}

/** your write 'a' to the fifo with check for existence & unlink */
int crt_fifo_write (char *fname)
{
    int fd, n = 0;
    errno = 0;

    if (!fname || !*fname) return 0;

    if (file_exists (fname))
        if (unlink (fname) == -1) {
            perror ("fifo exists unlink failed");
            return 0;
        }

    if (mkfifo (fname, 0666)) {
        perror ("mkfifo failed");
        return 1;
    }
    if ((fd = open (fname, O_WRONLY)) == -1) {
        perror ("open failed");
        return 1;    
    }

    printf ("Opened\n");
    char a = 'a';

    while (write (fd, &a, 1) != -1) {
        printf ("%3d - Writing to pipe...\n", n++);
    }

    return 0;
}

/** atomic test that file exists (1 success, 0 otherwise) */
int file_exists (char *f)
{
    errno = 0;
    int flags = O_CREAT | O_WRONLY | O_EXCL;
    int mode = S_IRUSR | S_IWUSR;
    int fd = open (f, flags, mode);

    if (fd < 0 && errno == EEXIST)
        return 1;
    else if (fd) {  /* created, like bash touch */
        close (fd);
        unlink (f);
    }

    return 0;
}

Example Program Use/Output

$ ./bin/pipe_write_shell_fork
Opened
  0 - Writing to pipe...
  1 - Writing to pipe...
  2 - Writing to pipe...
  3 - Writing to pipe...
  4 - Writing to pipe...
    ...
138 - Writing to pipe...
139 - Writing to pipe...
140 - Writing to pipe...
shell read complete. restarting
Opened
  0 - Writing to pipe...
  1 - Writing to pipe...
  2 - Writing to pipe...
  3 - Writing to pipe...
  4 - Writing to pipe...
    ...
    130 - Writing to pipe...
131 - Writing to pipe...
shell read complete. restarting
Opened
  0 - Writing to pipe...
  1 - Writing to pipe...
  2 - Writing to pipe...
  3 - Writing to pipe...
  4 - Writing to pipe...
    ...
144 - Writing to pipe...
145 - Writing to pipe...
shell read complete. done

Example Shell Read/Output

$ declare -i c=0; while test "$c" -lt 10 && read -n 8 ch; do \
echo "read: $ch"; ((c++)); done </tmp/test
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa
read: aaaaaaaa

(repeated 2 more times)

While this is pretty awesome, I'm still not sure how I can keep the code going after the loop finishes. I want it so I can re-execute your shell loop without having to restart the c program. — jrcichra, Jul 07 '17 at 17:27
That may take looking into a few options for the pipe itself. `mkfifo` generally requires *simultaneous* opening of both read/write ends. When the shell is done reading it effectively closes the read end, telling your C-program it is done with a `SIGPIPE` as William indicates. You will have to catch and handle that signal within your C program and most likely need to remove and reinitialize the fifo. — David C. Rankin, Jul 07 '17 at 17:42
Interesting, Java appears to handle pipes in a different sort of way, maybe because of it being mutli-platform? I can put data into a pipe with java and it will wait until somebody reads from the output pipe before it continues on. — jrcichra, Jul 07 '17 at 17:46
It seems your best bet would be a program that uses `fork` to setup the `fifo` and `write` as above. The parent can `waitpid` on the child to terminate and then spawn another child process and repeat. That way the child will terminate on each `SIGPIPE` and the parent will replace it with a new process waiting to write once the next shell process initiates the read. There may be other ways to handle it, but this was the first one to come to mind. — David C. Rankin, Jul 07 '17 at 17:53
@Demodude123 `man 2 wait` has a good example of the `fork/waitpid` framework you could use in a loop. You will need to check for the existence of `/tmp/test` (or whatever name you use), remove it if it exists, recreate the fifo with `mkfifo` and setup the `write`. A new process would be created on close of the read end by the shell. (which essentially looks like what your java code is doing, it is just handling the details behind the scene) — David C. Rankin, Jul 07 '17 at 18:03
Okay, but how does C know when somebody's on the other end of the pipe? The shell makes the request to the pipe for data. Only way I can think of telling is polling the pipe, "Is there somebody reading this pipe? Yes -> send them an a'. Your way is a solid idea which also may allow for multiple simultaneous 'a's to be sent out. Maybe I'm reading too much into this :D Which came first, the read or the write? :P — jrcichra, Jul 07 '17 at 18:13
It doesn't -- it doesn't care, the parent starts the child and it just blocks on write until somebody answers, it finishes that conversation and terminates and the next child process is created and starts blocking on write until the next somebody answers. The parent by spawning the children and waiting until they finish before spawning the next just insures your code is always blocking on write just waiting for the next shell to read. — David C. Rankin, Jul 07 '17 at 18:19
Looks like your idea works, I don't know what I was thinking. I made a rough cut version of it in my main post. Thanks! — jrcichra, Jul 07 '17 at 18:42
@Demodude123 - I added a quick example of the `fork` routine. Give it a look. — David C. Rankin, Jul 07 '17 at 19:28

C -> Shell - Block on a Write Until Read

UPDATE

UPDATE #2 - THIS IS THE SOLUTION

2 Answers2