Which is the suitable approach to handle multiple jobs in parallel without blocking the main thread

Question

I have a requirement where single process should handle multiple jobs in parallel. Each of these jobs should be run periodically (example, every 10 seconds). Also Main thread needs to watch for stop signal and when received should stop all the threads and exit.

Below is my approach to handle this requirement.

Main thread will create thread for each job and wait for stop signal.Each thread is responsible for handling the looping mechanism. When main thread receives stop signal, it will send a signal to the threads to stop.

In this mechanism the treads will be running all the time. So I was thinking there may be better approach to handle this. Maybe like, main thread will keep track of when each job should be executed next and will start the thread when required. Thread will perform some operation and exit on completion. In this case threads will not be running all the time.

I currently don't know how the alternate approach can be implemented. So, I wanted to know which of the above might be good approach? Also, if there are any other better alternatives, please do suggest them.

Edit [19/Mar]:

Maybe I should have mentioned this initially but here goes.

Say if I have 2 jobs, both need not be run at the same time. For example, Job 1 should run every 10 seconds & Job 2 should run every 20 second.

Also, if the Job itself takes more time, then there must be some mechanism to identify and handle it properly. Maybe skip the execution or wait for previous job to complete & then start again. Currently, the requirement is not clear on this one. But, I should be able to recognize and handle this situation.

Answer 1

Here are some suggestions.

First have the main thread spin up threads to complete tasks as necessary. The main thread detaches a thread when it creates it and then completely forgets about it. The detached thread will clean itself when it returns. This can easily simplify the main thread and also removed overhead from keeping track and managing different threads. This works if the threads are almost completely independent and will never encounter an infinite loop. The main thread will just exit when it receives the stop signal which in turn kills the process, however that depends on OS so check out What happens to a detached thread when main() exits?.

You spin up a thread that acts as a task/thread manager. This object/thread would create new threads as necessary and correctly wait for them via some pooling or thread tracking. Then the main thread can easily message the thread tracker to stop via a mutex guarded flag, on which case the thread tracker simply waits for all its spawned threads and then dies. This requires more overhead then the above solution but it gives you more information on which threads are running when. Also you have more control on if you need to kill a thread directly or how to message a thread if necessary. Also it allows for safe clean-up, which depending on your OS, threads may be cut short by the process dying. This is better if the threads need to be able to receive messages easier and if you want to ensure a thread runs to completion even after the stop signal (think R/W operations).

You could also mix the main thread and the thread/manager into one but this makes the main thread more complicated and does not reduce overhead signifcantly for most general purpose scenarios.

Answer 2

Take a look at the following class which runs a function at given intervalls:

class CallbackTimer
{
public:
    ~CallbackTimer()
    {
        stop();
    }

    void start(std::chrono::milliseconds interval, std::function<void()> callback)
    {
        stop();
        shouldQuit = false;

        handle = std::async([=,callback=std::move(callback)]() {

            while (!shouldQuit)
            {
                auto nextStart = std::chrono::steady_clock::now() + interval;
                callback();
                std::this_thread::sleep_until(nextStart);
            }
        });
    }

    void stop()
    {
        if (handle.valid())
        {
            shouldQuit = true;
            handle.get();
        }
    }

private:
    std::atomic_bool shouldQuit;
    std::future<void> handle;
};

On start() a new thread is created which runs the given 'callback' at given interval. For a 10 second intervall, the callback is called exactly every 10 seconds unless it's still working when the next start is due. In this case it's run again immediatly.

stop() sets the quit-flag and waits for the thread to quit. This very simple implementation of the main routing does not interrupt the sleep for shouldQuit-checks, so it may take up to one complete interval for the CallbackTimer to quit.

Be aware that callback times may slightly drift, so if you start two of them at the same intervall they may not be run concurrently after a time period. But the drift should be minimal and if its critical you should consider another solution.

Here is an usage example:

int main()
{
    CallbackTimer a;
    a.start(std::chrono::seconds(1), []() { std::cout << "a" << std::endl; });

    CallbackTimer b;
    b.start(std::chrono::seconds(2), []() { std::cout << "b" << std::endl; });

    std::this_thread::sleep_for(std::chrono::seconds(10));

    a.stop();
    b.stop();

    return 0;
}

For a quicker stop() you could implement a "beginStop()" method which only sets the 'shouldQuit' Flag to true and call this for all Instances before calling "stop()" on all instances. This way the shutdown of the second instance is not delayed until the shutdown of the first instance is complete.

Answer 3

If your main thread only has to manage the threads, you could just start new threads every 10 seconds and wait for them to complete.

The following example runs two threads in parallel, waits for them to complete and waits for an additional second before looping. (The shouldQuit() function uses compiler specific functions from Visual C++; perhaps you have to insert your own code there)

#include <future>
#include <iostream>
#include <conio.h>
#include <thread>
#include <chrono>

int a()
{
    std::cout << 'a' << std::flush;
    return 1;
}

int b()
{
    std::cout << 'b' << std::flush;
    return 2;
}

bool shouldQuit()
{
    while (_kbhit())
    {
        if (_getch() == 27)
            return true;
    }
    return false;
}

int main()
{
    auto threadFunctions = { a, b };

    while(!shouldQuit())
    {
        auto threadFutures = std::vector<std::future<int>>{};

        // Run asynchronous tasks
        for (auto& threadFunction: threadFunctions)
            threadFutures.push_back(std::async(threadFunction));

        // Wait for all tasks to complete
        for (auto& threadFuture : threadFutures)
            threadFuture.get();

        // Wait a second
        std::this_thread::sleep_for(std::chrono::seconds(1));
    }

    return 0;
}

The time consumed by the threads (while we wait for them to complete) is not part of the timing. The pause between the thread calls will be 1 second for the longest running thread and at least one second for all other threads.

Note: Running a new thread every few seconds is not the most efficient way of doing things. Creating a new thread is an expensive operation on most operating systems. If performance is critical you should reuse existing threads as you described in your question. But the overall complexity of the programm will increase.

Using std::async and std::future has the benefit of handling exceptions correctly. If a thread throws an exception, this exception is caught and stored in the std::future. It is rethrown in the thread calling future.get(). So if your threads may throw exceptions, better wrap your future.get() call in a try .. catch.

The future.get() call synchronizes the calling thread with the thread the future was created for. If the thread is already done, it simply returns the return value. If the thread is still working, the calling thread is blocked until the thread is done.