does a C++ piped system call always end with EOF?

Question

Using the SO post here: How do I read the results of a system() call in C++?

I was able to write a function that runs an arbitrary system command and returns any output as a string:

string pipeAndGetResults(string cmd)
{
    const char* ccmd = cmd.c_str();
    FILE* stream = popen( ccmd, "r" );
    std::ostringstream output;
    while( !feof( stream ) && !ferror( stream ))
    {
        char buf[128];
        int bytesRead = fread( buf, 1, 128, stream );
        output.write( buf, bytesRead );
    }
    string result = output.str();
    boost::trim(result);
    return result;
}

I have always used this for system commands that "instantly" produce a value. My question is whether this function will also work if cmd takes some time to run, say a minute, and then writes the results. I had problems doing something similar with Python's pexpect; it was timing out while waiting for a result if cmd took awhile, and I could not upper bound the runtime of cmd. I believe that question simplifies to whether cmd always writes eof after however long it takes to run?

score 3 · Answer 1 · answered Nov 05 '13 at 19:07

3

feof() does not detect an EOF character in the input stream. It indicates whether a read was attempted past the end of the file. Additionally, in my experience, I'd say that most commands do not write an EOF character at the end of their output. The fread() call will block until there is data to read, unless it is interrupted, so it doesn't matter how long the command runs. depending on which operating system you are using, you may be able to tell the system to resume the interrupted system call in the event of a signal. I also agree with Bastile. You should use a bigger buffer for more efficient I/O.

answered Nov 05 '13 at 19:07

Elkvis

728
1
5
21

Often (on x86) `char` are `unsigned` and `EOF` is -1 so cannot even be a `char` – Basile Starynkevitch Nov 05 '13 at 19:09
In particular, `EOF` is not part of the stream, but just a non-char integer returned by the standard C function `getchar()` to indicate that the stream has ended. Pipes cannot __end__ unless the operating system is quite sure that no further output will be written, which is no sooner than when the program whose output is being piped has terminated, or the program explicitly has closed its standard output stream. – Robert Jørgensgaard Engdahl Nov 05 '13 at 19:16
What do I do with the 1 in `fread` assuming I replace "128" with "4096"? – Tommy Nov 05 '13 at 19:20
1

The 1 can stay. It means to read 1 record. Changing the 128 to 4096 or sizeof(buf) tells it to read 1 record of _up to_ 4096 bytes. – Elkvis Nov 05 '13 at 20:26

Basile Starynkevitch · Accepted Answer · 2013-11-06T03:22:38.587

2

Unless that function is interrupted, it will run as you expect. BTW you buf is quite small (only 128 bytes). I would suggest 4 or 8 kilobytes (see getconf(1) with PIPE_BUF):

while( !feof( stream ) && !ferror( stream ))
{
    char buf[4096];
    memset (buf, 0, sizeof(buf)); //probably useless
    int bytesRead = fread(buf, 1, sizeof(buf), stream);
    if (bytesRead < 0) 
       break;
    output.write(buf,bytesRead);
}

Read also Advanced Linux Programming ... If you want to read and write from the same process, you'll probably need to multiplex e.g. with poll(2) after having created the pipe(2)-s.... etc etc..

^{The zeroing of buf with memset is probably not really needed; however, if the above program has bugs, it will make it much more reproducible. This is why I like zeroing memory. and here zeroing memory is much faster than reading it.}

does a C++ piped system call always end with EOF?

2 Answers2

Linked