c++: subprocess output to stdin

Question

Suppose I want to call a subprocess from within my program, and I want to read the output from that subprocess into my program.

Here is a trivial way to do that:

//somefile.cpp
system("sub_process arg1 arg2 -o file.out");
           //call the subprocess and have it write to file
FILE *f = std::fopen("file.out", "r");
//.... and so on

We all know that i/o operations are computationally slow. To speed this up, I would like to skip the write-to-file-then-read-from-file step, and instead redirect the output of this sub-process directly into stdin (or some other stream)

How would I do this? How do I skip the i/o operation?

Note: many programs spit out some diagnostic stuff into stdout while they run, and write a clean version of the output to stdout (ex: stdout: "step1...done, step2...done, step3..done" -o file-out: "The magic number is: 47.28"), so ignoring the "-o " argument and trusting that output will be automatically re-directed to stdout isn't necessarily helpful...

Thanks to all in advance.

Regarding your comment about -o and what happens there - most programs will have a strict definition of what they will output, which your program can filter. — Dan, Dec 08 '11 at 22:04
+CycoMatto, Did you mean "some diagnostic stuff into **stderr** while they run, and write a clean version of the output to **stdout**." Consider a program that outputs some output to '-o file', some output to **stdout**, and some output to **stderr**. Which of those three do you wish to have access to? — Aaron McDaid, Dec 08 '11 at 22:46
@AaronMcDaid Yeah, this was my crucial misunderstanding. Thanks for the clarification! See my response to dasblinkenlight below.. — cmo, Dec 08 '11 at 22:55

score 8 · Accepted Answer · edited Aug 30 '19 at 17:43

8

Using popen skips the file, and gets you command's output through an in-memory buffer.

#include <iomanip>
#include <iostream>
using namespace std;
const int MAX_BUFFER = 255;
int main() {
    string stdout;
    char buffer[MAX_BUFFER];
    FILE *stream = popen("command", "r");
    while ( fgets(buffer, MAX_BUFFER, stream) != NULL )
        stdout.append(buffer);
    pclose(stream);
    cout << endl << "output: " << endl << stdout << endl;
}

edited Aug 30 '19 at 17:43

kalehmann

4,821
6
26
36

answered Dec 08 '11 at 22:04

Sergey Kalinichenko

714,442
84
1,110
1,523

I'm guessing the output is quite large, and will take up much more than 255 bytes. +CycoMatto , can you clarify how large you expect this output to be? – Aaron McDaid Dec 08 '11 at 22:29
@AaronMcDaid that's OK - the program will read the output in 256-byte increments. If that turns out to be a performance issue, one can always change the size of the buffer. I prefer starting small, and act when the profiler tells me it's a bottleneck. – Sergey Kalinichenko Dec 08 '11 at 22:35
But as I mentioned, the command's output will include diagnostic garbage, won't it? I am looking for the lean, clean stuff written to file via the -o command. i.e, I would like to do "-o stdin" – cmo Dec 08 '11 at 22:36
1

@CycoMatto You will get only what goes out to stdout, none of the "debugging junk" that's normally sent to stderr. You see the two mixed, because they go to the same console, but internally they are separate streams. If the process that you are planning to run sends "stray output" to `stdout`, you will have to filter it yourself. But most UNIX programs are good players in this regard, so I think you should give it a try. – Sergey Kalinichenko Dec 08 '11 at 22:43
+CycoMatto, that is possible. Bear with me while I try to write it up properly. `mkfifo` is your friend. (Update: I'll wait a moment, I want more clarify, see my comment on the question) – Aaron McDaid Dec 08 '11 at 22:44
@dasblinkenlight Aaah, yes that is my misunderstanding. I overlooked the fact that stderr and stdout are both displayed in the same console. I suppose I assumed that everybody was irresponsible like I am and wrote everything to `stdout`. I will correct my etiquette immediately ;). Thank you! – cmo Dec 08 '11 at 22:55
1

I've often thought that the shell/terminal/whatever should find a way to colorize the stderr, precisely so that this confusion won't happen. – Aaron McDaid Dec 08 '11 at 22:58
any idea what the namespace of `popen` and `pclose` are? They are not in namespace `std` , although supposedly they come from `stdio`... – cmo Dec 08 '11 at 23:23
2

@CycoMatto `popen` is a system call of UNIX, not a C/C++ library function. That is why it is not part of a namespace. – Sergey Kalinichenko Dec 09 '11 at 00:47

score 3 · Answer 2 · answered Dec 08 '11 at 22:39

3

If you happen to be on windows :

follow this : http://support.microsoft.com/kb/190351

It describes it better than I ever would. You can redirect everything, everywhere.

answered Dec 08 '11 at 22:39

FailedDev

26,680
9
53
73

c++: subprocess output to stdin

2 Answers2

Linked

Related