Calling popen() pipe stream with a loop shell command line

Question

I'm testing this example for programming with pipes and it seems pretty straightforward.

But I was wondering what happen if the first argument of first popen() call (the string containing a shell command) holds a 'while do' loop.

For example, if I execute this shell command for 3 seconds, I get this output:

tomas@ubuntu64:~$ while true; do ps -A; sleep 1; done | grep init
    1 ?        00:00:03 init
    1 ?        00:00:03 init
    1 ?        00:00:03 init

so the grep is working in each iteration.

However, if I do it through the C language example, changing the popen() of the example by:

FILE* ps_pipe = popen("while true; do ps -A; sleep 1; done", "r");

I get no output result when executing the compiled C program.

Anyone can shed some light on this?

Did you actually read from that stream? What did you do with the output you read, and what did you expect to see? It's hard to tell what might be going on without a complete self-contained program. — Useless, Oct 02 '14 at 13:24

Serge Ballesta · Answer 1 · 2014-10-02T16:05:40.683

2

Edit: As noticed by J.F. Sebastian, by default grep uses large buffers when output is not directed to a terminal. You need to use option --line-buffered to get output immediately (after each line)

Well, I tried it and it works fine (thank to the fix of J.F. Sebastian). Here's full code on a FreeBSD 9 box :

#include <stdio.h>

int main() {

    char buffer[256];

    FILE *fd = popen("while true; do ps -A; sleep 1; done | grep --line-buffered init", "r");

        while(NULL != fgets(buffer, sizeof(buffer), fd)) {
            printf("GOT >%s<\n", buffer);
        }

    return 0;

}

And (as I did not remove the \n at end of buffer) the output is :

GOT >   1 ??  ILs    0:00:02 /sbin/init --
>
GOT >1334 v0  R+     0:00:00 grep --line-buffered init
>
GOT >   1 ??  ILs    0:00:02 /sbin/init --
>
GOT >1334 v0  R+     0:00:00 grep --line-buffered init
>

edited Oct 02 '14 at 16:05

answered Oct 02 '14 at 13:31

Serge Ballesta

143,923
11
122
252

@J.F.Sebastian I know it is not the same command, but it works the same ... and I'm too lazy to run it again. Feel free to edit my answer with exact command and related output, or to post your own. – Serge Ballesta Oct 02 '14 at 15:48
I've already posted [my answer](http://stackoverflow.com/a/26164479/4279). And I **can reproduce** the behaviour described in the question. Adding `grep` **does** make difference on my machine (Ubuntu). – jfs Oct 02 '14 at 15:49
@J.F.Sebastian: oups you are right ! I thing I need to edit this one :-( Thanks a lot for noticing ... – Serge Ballesta Oct 02 '14 at 15:51

score 1 · Answer 2 · edited May 23 '17 at 12:05

1

It is a block-buffering issue. When grep's stdout is not a terminal (tty) e.g., when it is a pipe (created by popen()) as in your case; it uses block buffering instead of line buffering. You won't see anything until grep's stdout buffer overflows. Add --line-buffered parameter to grep and you'll see the output immediately (each second).

edited May 23 '17 at 12:05

Community

1
1

answered Oct 02 '14 at 15:43

jfs

399,953
195
994
1,670

OP had problem simply running `FILE* ps_pipe = popen("while true; do ps -A; sleep 1; done", "r");`, he wasn't facing any issue with `grep` (are seeing any thing related `grep` here?!), he was just trying to experiment *first* time ever with `popen` and immediately faced problem of not getting the output. However you missed that and -1 for that! – rakib_ Oct 02 '14 at 16:31
@rakib: OP says: *"I get no output result when executing the compiled C program."* but if you run the command *without grep* then there is no block-buffering issue (at least on my machine) and OP should be able to see the result. On the other hand if I use `grep` then I get no result (the same as OP). That is why the presence of grep is significant. I assume that OP copy-pasted his code incorrectly i.e., there *is* `grep` in his actual code. – jfs Oct 02 '14 at 16:35
OP didn't wasn't trying with `grep` inside `popen()`. Why don't you understand this? You have misread the question and added the explanation of `--line-buffered` vaguely. Please *reread* the question again and try to address the problem OP is facing, don't add your superficial thoughts. – rakib_ Oct 02 '14 at 16:40
Why I would reread the last sentence? Your comment or what you superficially thinks don't matter to me, it's the question that we are talking about and this is what it should be. On my answer, you will find OP's comment regarding his confusion (which indicates that he is trying to learn the concept) about the concept, despite that you are trying to inject your own thought ! – rakib_ Oct 02 '14 at 16:54

score -1 · Answer 3 · answered Oct 02 '14 at 13:20

-1

All you need to do is to write the output to standard I/O, therefore change the popen() parameters to the following:

          FILE* ps_pipe = popen("while true; do ps -A; sleep 1; done", "w");

answered Oct 02 '14 at 13:20

rakib_

136,911
4
20
26

I think OP is trying to _read_ the output from that shell command, not write to its input. – Useless Oct 02 '14 at 13:25
@Useless I think OP says "I get no output result when executing the compiled C program." It's about output, or I have missed it? – rakib_ Oct 02 '14 at 13:26
Ok, this makes it work... But I'm afraid I don't understand why... I think I need to study a bit more 8-) – nephewtom Oct 02 '14 at 14:41
2

This will work if you never actually read the program's output in your version, because it now shares your parent process' stdout. If you wanted to read those lines into your parent process, you need to popen for reading (as you did) and then actually read. – Useless Oct 02 '14 at 14:57
@J.F.Sebastian Thanks for your insight , I guess I'm aware of what reading, writing are all about. But did you picked the questions correctly? – rakib_ Oct 02 '14 at 16:32
@Useless Yes, you are right. But, it's quite apparent that OP is having problem running his simple command of his own and he don't have any clue. And this answer will help him to get the output as he wants, he didn't mention what particular way he wants (why would he go for any complex way of getting it, when he's having problem to get it straight?). – rakib_ Oct 02 '14 at 16:59
With your solution OP isn't programming with pipes in any meaningful sense - this just forks & execs a subprocess and lets it inherit stdout. – Useless Oct 02 '14 at 18:19
@Useless Ask OP, not me. I'm seeing you are having serious problem in understanding that, OP is just learning and he is confused, take a look at his comment following my answer. Even that isn't clear to you, then just Goodbye. – rakib_ Oct 03 '14 at 00:42

Calling popen() pipe stream with a loop shell command line

3 Answers3