0

I already have a solution to my problem, but I am trying to figure out what exactly is going on. I dont know why the solution solves my problem.

A few days ago SO helped me with a function to results: does a C++ piped system call always end with EOF?

I reproduce the code here:

string pipeAndGetResults(string cmd)
{
    const char* ccmd = cmd.c_str();
    FILE* stream = popen( ccmd, "r" );
    std::ostringstream output;
    while( !feof( stream ) && !ferror( stream ))
    {
        char buf[128];
        int bytesRead = fread( buf, 1, 128, stream );
        output.write( buf, bytesRead );
    }
    string result = output.str();
    boost::trim(result);
    return result;
}

I had the strangest error occurring. I was running that in a loop which calls some process with different data each time in the loop. On my Ubuntu box, I could run this 200 or 300 times (never tried more) without problem. However, on my mac, this bombs on iteration 77 every time. This was a hard bug to figure out, because popen doesnet work well if the subprocess bombs since exceptions can't be caught across processes.

It seems that if I modify the code by appending:

    pclose(stream);

right before the return statement, I can run this many times without any errors.

I need to mention that if I look at activity monitor while this is running, I do not see 78 processes even without pclose. So these can't be processes. They have no PIDs.

So my question is, what is going on here? I was under the impression that if the subproccess just runs and prints something, then everything would kind of just clean up after this loop exits (btw, no one in that other post mentioned pclose). Moreover, if a pipe is remaining open to the same process or stdout file, why does it work for iteration 18, or 27? Why not bomb on iteration 2 when stuff is still open that shouldn't be?

Any help in helping me understand is greatly appreciated.

Community
  • 1
  • 1
Tommy
  • 12,588
  • 14
  • 59
  • 110
  • 1
    I would guess you are running out of file descriptors. If you don't pclose in that loop you can chew through them. But 77 seems awfully low unless you are doing similar things in other functions. – Duck Nov 07 '13 at 02:03
  • The macosx man age for pclose says it calls wait4. It may be the case that if you don't call pclose, your process won't wait for those popened processes and they will just pile up as zombies, 2 or 3 per popen, until the per-user process limit is reached. – Mark Plotnick Nov 07 '13 at 02:19
  • I should mention that the process called using open opens a file handle to a file, hence, in a loop, a handle to the same file each time. However, I would expect this handle to die when the process finishes, which I thought happened as soon as it terminates (which happens regardless of whether pclose is called). Also, like you said, 77 seems super low. – Tommy Nov 07 '13 at 03:02

0 Answers0