use popen without blocking

Question

In C++ I am running a bash command. The command is "echo | openssl s_client -connect zellowork.io:443"

But if this fails I want it to timeout in 4 seconds. The typical "/usr/bin/timeout 4 /usr/bin/sh -c" before the command does not work when run from the c++ code.

So I was trying to make a function that uses popen to send out the command and then waits for up to 4 seconds for the command to complete before it returns. The difficulty that I have is that fgets is blocking and it will wait for 20 seconds (on this command) before it unblocks and fails and I can not find anyway to see if there is something to read in a stream before I call fgets. Here is my code.

ExecuteCmdReturn Utils::executeCmdWithTimeout(string cmd, int ms)
{
    ExecuteCmdReturn ecr;
    ecr.success = false;
    ecr.outstr = "";

    FILE *in;
    char buff[4096];

    u64_t startTime = TWTime::ticksSinceStart();
    u64_t stopTime = startTime + ms;

    if(!(in = popen(cmd.c_str(), "r"))){
        return ecr;
    }  
    fseek(in,0,SEEK_SET);  

    stringstream ss("");
    long int lastPos = 0;
    long int newPos = 0;
    while (TWTime::ticksSinceStart() < stopTime) {
        newPos = ftell(in);
        if (newPos > lastPos) {
            lastPos = newPos;
            if (fgets(buff, sizeof(buff), in) == NULL) {
                break;
            } else {
                ss << buff;
            }
        } else {
            msSleep(10);
        }
    }

    auto rc = pclose(in);

    ecr.success = true;
    ecr.outstr = ss.str();
    return ecr;
}

Would making `popen` non-blocking help? https://stackoverflow.com/questions/1735781/non-blocking-pipe-using-popen — Galik, Jul 12 '21 at 15:20
You probably need to look into POSIX timer functions and signals. — Jonathan Leffler, Jul 12 '21 at 15:49
making popen non blocking does help, But then when you do pclose it still waits for the process to end before it stops blocking. So I still can't terminate the process for 20 seconds. So even if you stop blocking on the popen it still blocks on pclose...that seems stupid. — user846566, Jul 12 '21 at 16:03

AndyG · Accepted Answer · 2021-07-12T16:13:26.147

Use std::async to express that you may get your result asynchronously (a std::future<ExecuteCmdReturn>)
Use std::future<T>::wait_for to timeout waiting for the result.

Here's an example:

First, a surrogate for your executeCmdWithTimeout function that randomly sleeps between 0 and 5 seconds.

int do_something_silly()
{
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_int_distribution<> distribution(0, 5);
    auto sleep_time = std::chrono::seconds(distribution(gen));
    std::cout << "Sleeping for " << sleep_time.count() << " seconds\n";
    std::this_thread::sleep_for(sleep_time);
    return 42;
}

Then, launching the task asynchronously and timing out on it:

int main()
{
    auto silly_result = std::async(std::launch::async, [](){ return do_something_silly();});
    auto future_status = silly_result.wait_for(3s);
    switch(future_status)
    {
        case std::future_status::timeout:
            std::cout << "timed out\n";
            break;
        case std::future_status::ready:
            std::cout << "finished. Result is " << silly_result.get() << std::endl;
            break;
        case std::future_status::deferred:
            std::cout << "The function hasn't even started yet.\n";
    }
}

I used a lambda here even though I didn't need to because in your situation it will be easier because it looks like you are using a member function and you'll want to capture [this].

Live Demo

In your case, main would become ExecuteCmdReturn Utils::executeCmdWithTimeout(string cmd, int ms) and do_something_silly would become a private helper, named something like executeCmdWithTimeout_impl.

If you timeout waiting for the process to complete, you optionally kill the process so that you aren't wasting any extra cycles.

If you find yourself creating many short-lived threads like this, consider thread pooling. I've had a lot of success with boost::thread_pool (and if you end up going that direction, consider using Boost.Process for handling your process creation).

This is really great! Is there anyway to kill "do_something_silly"? I need to keep trying a few times in a row. "do_something_silly" may need to be called again while it is still busy? — user846566, Jul 12 '21 at 19:12
There is a technique called "cooperative cancellation" which will require a bit more work on your part, and since it's a thread and not a process, cancellation is never instantaneous (and often not even guaranteed because the thread may be hung). C++20 introduces std::jthread that implements this technique, but the gist is that you need to use a loop in your thread to periodically check for cancellation (which can be represented in a number of ways, e.g. an atomic bool) — AndyG, Jul 12 '21 at 19:30
A better approach would be to make the function pure so that multiple may be in flight at once. Basically, if state transformation is needed, the function accepts a copy of state (e.g. a simple data struct) and returns an updated copy. — AndyG, Jul 12 '21 at 19:33
@user846566: Can you let me know what more you need before you mark this answer as accepted? — AndyG, Jul 13 '21 at 19:00

use popen without blocking

1 Answers1

Live Demo