C++ popen pipe doesn't close correctly with pclose for "ps aux" command

Question

I am running MacOS and want to execute the "ps aux" command and get its output through my application. I have written a method which executes commands using popen function:

std::string exec(const char* cmd) {

    char buffer[128];
    std::string result = "";

    FILE* pipe = popen(cmd, "r");
    if (!pipe) throw std::runtime_error("popen() failed!2");
    try {
        while (!feof(pipe)) {
            if (fgets(buffer, 128, pipe) != NULL)
                result += buffer;
        }
    } catch (...) {
        pclose(pipe);

        throw;
    }
    pclose(pipe);


    return result;
}

I have a loop that is constantly running the exec("ps aux") function. The problem is that the pipe from popen is not closed which I have checked using "lsof" command from the terminal. After 20 or so seconds, there are like 300 opened file descriptors by the application which prevents the application from opening more pipes (running the "ps aux" command) from the loop.

What I have discovered, is that the exec function works fine for other commands (the pipe gets closed correctly), for example "netstat", so it must be something in the "ps aux" command that's preventing the pipe to close.

I have searched a lot about that issue, but haven't found any solution. Can somebody please point me in the right direction?

Thank you!

*Possibly* unrelated to your problem, but please take some time to read [Why is “while ( !feof (file) )” always wrong?](https://stackoverflow.com/questions/5431941/why-is-while-feof-file-always-wrong) — Some programmer dude, Feb 20 '18 at 18:10
Also unrelated - but I personally would use an RAII class instead of an explicit catch/rethrow, like maybe `struct PipeCloser { void operator()(FILE* f) { pclose(f); } }; using PipeUniquePtr = std::unique_ptr;` — Daniel Schepler, Feb 20 '18 at 18:33

score 0 · Answer 1 · answered Feb 20 '18 at 18:49

0

I can't see what is specifically wrong with your code. For these things I use a custom deleter with a std::unique_ptr to make sure the file closes at all possible exits.

Also note it is not recommended to loop using while(eof(...)) for a few reasons. One is eof is not set in the event of an error. More info here.

// RAII piped FILE*

// custom deleter for unique_ptr
struct piped_file_closer
{
    void operator()(std::FILE* fp) const { pclose(fp); }
};

// custom unique_ptr for piped FILE*
using unique_PIPE_handle = std::unique_ptr<std::FILE, piped_file_closer>;

//
unique_PIPE_handle open_piped_command(std::string const& cmd, char const* mode)
{
    auto p = popen(cmd.c_str(), mode);

    if(!p)
        throw std::runtime_error(std::strerror(errno));

    return unique_PIPE_handle{p};
}

// exception safe piped reading
std::string piped_read(std::string const& cmd)
{
    std::string output;

    if(auto pipe = open_piped_command(cmd, "r"))
    {
        char buf[512];
        while(auto len = std::fread(buf, sizeof(char), sizeof(buf), pipe.get()))
            output.append(buf, len);

        if(std::ferror(pipe.get()))
            throw std::runtime_error("error reading from pipe");
    }

    return output;
}

Calling auto output = piped_read("ps aux"); hundreds of times on my system does not produce your error with this code.

answered Feb 20 '18 at 18:49

Galik

47,303
4
80
117

Thank you for the help! However, using that code I still get that problem - the application crashes with that message: "Application Specific Information: abort() called terminating with uncaught exception of type std::runtime_error: Too many open files" – MrWhite Feb 20 '18 at 19:02
Running it hundreds of times won't cause the application to crash, but running it 300 times definitely would because the default limit of opened file descriptors per process on MacOS is 256 if I remember correctly. – MrWhite Feb 20 '18 at 19:04
@MrWhite After every call the file descriptor for that call should be closed so you should be able to run the command as many times as you like. My guess is that your problem is somewhere else? Maybe some subtle undefined behavior? I am now seriously stress testing this code and the file descriptor count is not changing whilst continually looping your command. – Galik Feb 20 '18 at 19:08
1

@MrWhite Now I am stress testing *your* code and getting the same results, no problems on my machine. Perhaps the code you posted if fine and the problem is something else? – Galik Feb 20 '18 at 19:13
1

I am so sorry, I wasn't closing a socket which was opened in the loop if the "ps aux" command was successfully executed so that's what caused the number of file descriptors to go up :/ – MrWhite Feb 21 '18 at 11:25

C++ popen pipe doesn't close correctly with pclose for "ps aux" command

1 Answers1