popen not working as expected

Question

I have a program where I need to communicate with another program on its standard input and output. I can use a pipe to send it input and redirect its output to a specified file to be read when it is done, and that works fine. I am not using pipe and dup2 because I would like my program to at least somewhat work on windows as well as linux.

My issue is that I want to use the pipes to stream data.

I tried the following snippet:

#include <stdio.h>

int main()
{
    while (!feof(stdin)) {
        int x;
        scanf("%d", &x);
        printf("%x ", x);
        fflush(0);
    }

    return 0;
}

in conjunction with:

#include <unistd.h>
#include <stdio.h>

int main()
{
        int x;
    FILE *p = popen("./test > tmp.datafile", "w");
    FILE *f = fopen("tmp.datafile", "r");

    for (int i = 0; i < 100; ++i) {
        fprintf(p, "%d", i);

        sleep(1);
        x = fgetc(f);
        printf("%c ", x);
        fflush(0);
    }

    fclose(f);
    pclose(p);

    return 0;
}

However no matter what I do, I just keep getting nulls out of the file. I suspect that perhaps there are concurrency issues, in that I try to read the file before the test executable finishes flushing it, however I am not sure how to fix this.

Is there a better way to communicate with a program via standard streams in c?

Actually, you should use [poll(2)](http://man7.org/linux/man-pages/man2/poll.2.html) in some [event loop](https://en.wikipedia.org/wiki/Event_loop). Read http://advancedlinuxprogramming.com/ and [this](http://stackoverflow.com/a/20582916/841108) answer. — Basile Starynkevitch, Dec 01 '16 at 18:16
I hadn't thought about poll'ing the file, that would also work. Do you know how similar windows poll is to linux? I know that window popen is nearly identical, but not sure about poll. — LambdaBeta, Dec 01 '16 at 18:17
I don't know Windows at all, but some libraries like [POCO](http://pocoproject.org/) or [Qt](http://qt.io/) or [GLIB](https://developer.gnome.org/glib/stable/) (or perhaps [libevent](http://libevent.org/)....) should help and are cross-platform. You might want [named pipes](https://en.wikipedia.org/wiki/Named_pipe) — Basile Starynkevitch, Dec 01 '16 at 18:19

score 3 · Accepted Answer · answered Dec 01 '16 at 18:09

fgetc() won't read past EOF once EOF has been reached — which may happen on the very first read. Doing something like

if(feof(f)) {
    clearerr(f);
    sleep(1);
} else {
    printf("%c ", x);
}

should sort-of solve the issue with a busy loop.

A better idea would be to wait for the file to change once EOF has been reached, but that would require system-specific calls.

Beware of another race condition: fopen may happen before the file has been created by the command started via popen.

while(!(f = fopen(..., "r"))
    sleep(1);

It's another busy loop and overall not a good solution that should only be used if dup2() is not available.

Fair enough, I hadn't thought about the fact that I was reading past end. I think I'm going to go just support linux for now, and later add `#ifdef` hell to try to support windows. This will indeed be much easier with fork, dup2, execlp. — LambdaBeta, Dec 01 '16 at 18:11

score 0 · Answer 2 · answered Dec 01 '16 at 18:43

One of the issues in your code is that you are not writing any white space to the input of p, because of which the scanf() call in ./test is waiting for the input to finish so that it could read the value. Because of which the stdout of ./test remains empty. Fix it by something like this:

fprintf(p, "%d\n", i);

Also, consider using unbuffered I/O functions inside the ./test program. printf is part of the buffered I/O family of functions and the output might not be written immediately to the file.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>

int main()
{
    while (!feof(stdin)) {
        int x;
        char s;
        scanf("%d", &x);
        sprintf(&s, "%x", x);
        write(STDOUT_FILENO, &s, 1);
        fflush(stdout);
    }

    return 0;
}

popen not working as expected

2 Answers2