C++: Use future.get with timeout and without blocking

Question

I'm having this problem, where I have a main loop, that needs to trigger an async work and must not wait for it to finish. What I want it to do is to check every while-loop whether the async work is done. This can be accomplished with the future.wait_for().

Since I don't want to block the main loop, I can use future.wait_for(0). So far so good.

In addition, I'd like to verify that I received (or didn't receive) an answer within X ms. I can do that by checking how long since I launched the "async", and verify what comes first - X ms passed or future_status::ready returned.

My question - is this a good practice, or is there a better way to do it?

Some more information: Since the main loop must launch many different async jobs, it means I need to have a lot of duplicated code - every launch needs to "remember" the timestamp it was launched and every time I check if the async job is ready, I need to re-calculate the time differences for each async job. This might be quite a hassle.

for now - this is an example of what I described (might have build errors):

#define MAX_TIMEOUT_MS 30
bool myFunc()
{
    bool result = false;
    //do something for quite some time
    return result;
}

int main()
{
    int timeout_ms = MAX_TIMEOUT_MS;
    steady_clock::time_point start;
    bool async_return = false;
    std::future_status status = std::future_status::ready;
    int delta_ms = 0;

    while(true) {
        // On first time, or once we have an answer, launch async again
        if (status == std::future_status::ready) {
            std::future<bool> fut = std::async (std::launch::async, myFunc);
            start = steady_clock::now();        // record the start timestamp whenever we launch async()
        }

        // do something...

        status = fut.wait_for(std::chrono::seconds(0));
        // check how long since we launched async
        delta_ms = chrono::duration_cast<chrono::milliseconds>(steady_clock::now() - start).count();

        if (status != std::future_status::ready && delta_ms > timeout_ms ) {
            break;
        } else {
            async_return = fut.get();
            // and we do something with the result
        }
    }

    return 0;
}

Answer 1

One thing you might want to consider: If your while loop doesn't do any relevant work, and just checks for task completion, you may be doing a busy-wait (https://en.wikipedia.org/wiki/Busy_waiting). This means you are wasting a lot of CPU time doing useless work. This may sound counter-intuitive, but it can negatively affect your performance in evaluating task completion even if you are constantly checking it!

This can happen because this thread will look like it is doing a lot of work to the OS, and will receive high priority for processing. Which may make other threads (that are doing your async job) look less important and took longer to complete. Of course, this is not set in stone and anything can happen, but still, it is a waste of CPU if you are not doing any other work in that loop.

wait_for(0) is not the best option since it effectively breaks the execution of this thread, even if the work is not ready yet. And it may take longer than you expect for it to resume work (https://en.cppreference.com/w/cpp/thread/future/wait_for). std::future doesn't seem to have a truly non-blocking API yet (C++ async programming, how to not wait for future?), but you can use other resources such as a mutex and the try_lock (http://www.cplusplus.com/reference/mutex/try_lock/).

That said, if your loop still does important work, this flow is ok to use. But you might want to have a queue of completed jobs to check, instead of a single future. This queue would only be consumed by your main thread and can be implemented with a non-blocking thread-safe "try_get" call to get next completed jobs. As others commented, you may want to wrap your time-saving logic in a job dispatcher class or similar.

Maybe something like this (pseudo code!):

struct WorkInfo {
  time_type begin_at; // initialized on job dispatch
  time_type finished_at;
  // more info
};

thread_safe_vector<WorkInfo> finished_work;

void timed_worker_job() {
  info.begin_at = current_time();
  
  do_real_job_work();
  
  WorkInfo info;
  info.finished_at = current_time();
  finished_work.push(some_data);
}

void main() {
   ...
   
   while (app_loop)
   {
       dispatch_some_jobs();

       WorkInfo workTemp;
       while (finished_work.try_get(&work)) // returns true if returned work
       {
         handle_finished_job(workTemp);
       }
   }
   
   ...
}

And if you are not familiar, I also suggest you to read about Thread-Pools (https://en.wikipedia.org/wiki/Thread_pool) and Producer-Consumer (https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem).

Answer 2

The code below runs tasks async and checks later if they are finished.

I've added some fake work and waits to see the results.

#define MAX_TIMEOUT_MS 30

struct fun_t {
    size_t _count;
    bool finished;
    bool result;
    fun_t () : _count (9999), finished (false), result (false) {
    }
    fun_t (size_t c) : _count (c), finished (false), result (false) {
    }
    fun_t (const fun_t & f) : _count (f._count), finished (f.finished), result (f.result) {
    }
    fun_t (fun_t && f) : _count (f._count), finished (f.finished), result (f.result) {
    }
    ~fun_t () {
    }
    const fun_t & operator= (fun_t && f) {
        _count = f._count;
        finished = f.finished;
        result = f.result;
        return *this;
    }
    void run ()
    {
        for (int i = 0; i < 50; ++i) {
            cout << _count << " " << i << endl;;
            std::this_thread::sleep_for(std::chrono::milliseconds(100));
        }
        result = true;
        finished = true;
        cout << "       results: " << finished << ", " << result << endl;
    }
    operator bool () { return result; }
};

int main()
{
    int timeout_ms = MAX_TIMEOUT_MS;
    chrono::steady_clock::time_point start;
    bool async_return = false;
    std::future_status status = std::future_status::ready;
    int delta_ms = 0;
    std::map<size_t, fun_t> futs;
    std::vector<std::future<void>> futfuncs;

    size_t count = 0;
    bool loop = true;

    cout << "Begin --------------- " << endl;

    while (loop) {
        loop = false;
        // On first time, or once we have an answer, launch async again
        if (count < 3 && status == std::future_status::ready) {
            //std::future<bool> fut = std::async (std::launch::async, myFunc);
            futs[count] = std::move(fun_t(count));
            //futs[futs.size() - 1].fut = std::async (std::launch::async, futs[futs.size() - 1]);
            futfuncs.push_back (std::move(std::async(std::launch::async, &fun_t::run, &futs[count])));
        }

        // do something...
        std::this_thread::sleep_for(std::chrono::seconds(2));

        for (auto & f : futs) {
            if (! f.second.finished) {
                cout << " Not finished " << f.second._count << ", " << f.second.finished << endl;
                loop = true;
            } else {
                bool aret = f.second;
                cout << "Result: " << f.second._count << ", " << aret << endl;;
            }
        }

        ++count;
    }

    for (auto & f : futs) {
        cout << " Verify " << f.second._count << ", " << f.second.finished;
        if (f.second.finished) {
            bool aret = f.second;
            cout << "; result: " << aret;
        }
        cout << endl;
    }

    cout << "End --------------- " << endl;
    return 0;
}

After removing lines (there are too much) you see the tasks. First number is the task id, second the iteration number.

Begin --------------- 
0 0
0 1
0 2
 Not finished 0, 0
1 0
0 20
1 1 
 Not finished 0, 0
 Not finished 1, 0
2 0
1 20
0 40
2 1
0 49   // here task 0 ends
2 10
1 30
       results: 1, 1  // "run" function ends
1 39
Result: 0, 1         // this is the verification "for"
 Not finished 1, 0
 Not finished 2, 0
       results: 1, 1
Result: 0, 1
Result: 1, 1
Result: 2, 1
 Verify 0, 1; result: 1
 Verify 1, 1; result: 1
 Verify 2, 1; result: 1
End ---------------