Threadpool - CPU usage?

Question

I am working on a Windows C++ application. We use the boost library. I have an operation in my application that can be parallelized to run on multiple threads. Number of threads depends each time on the operation parameters and can be big(say like 50 or 70). I dont want to spawn the maximum threads that I can, since that is a risk of the application being non-responsive to other operations(since the all the processor(s) could be occupied doing this). How can I make sure I dont create a situation I described? Would a threadpool help and if so how?

If you think that creating 50 threads is a good idea then, yes, do heavily favor a thread pool. It is built into the OS, you'll like SetThreadpoolThreadMaximum(). — Hans Passant, Feb 22 '18 at 10:11

score 0 · Answer 1 · answered Feb 22 '18 at 13:15

0

Just create a thread pool, e.g. the one I posted here boost thread throwing exception "thread_resource_error: resource temporarily unavailable"

Two more flavours here c++ work queues with blocking (one using Asio, one using just C++11)

answered Feb 22 '18 at 13:15

sehe

374,641
47
450
633

score 0 · Accepted Answer · answered Feb 22 '18 at 14:27

70 threads on modern hardware can be easily handled w/o any noticeable impact on system performance. Thread creation time, memory usage, scheduling and context switch overhead can be a problem but we don't know if it's a problem in your particular case.

If creating 70 threads is not an option, consider using OpenMP (supported by all major compilers) as it's a very simple and often very efficient solution:

#pragma omp parallel for
for(int i = 1; i < 100; ++i)
{
   do_task(i);
}

It uses a thread pool under the hood.

If OpenMP is not acceptable for some reason(s), you can go with explicit thread pool. It can be a "home-made" thread pool (not recommended), or one from @sehe's answer, or one that is provided by OS (as @Hans Passant mentioned in his comment), or one from a 3rd-party library (e.g. Intel Threading Building Blocks).

Yes, thread pool can help with responsiveness, though typical thread pool implementation by default creates number of threads == number of logical CPU cores. This means all your cores can be busy doing your work and it's not necessarily a problem. Windows uses preemptive multithreading. This means it can handle number of threads much greater than number of CPUs and still being responsive.

Thread pool can help because it's not possible to simultaneously execute more tasks than number of logical CPU cores you have. Thread pool can be more efficient because of better use of caches and reduced number of context switches. Or because same threads can be used to execute your operation multiple times. To know for sure profile your performance.

Thanks @Andriy! I am thinking of using OpenMP. What would I do if some operation should not continue the full operation based on some condition? How do I return? — quickdraw, Feb 22 '18 at 16:36
i am planning to call a function(that does some extensive calculation or does not under some conditions when I would like to return) like you have mentioned in your code. When I tried writing a simple program to see how OpenMP works, it did not compile with a return inside the openmp parallel block, but when I made the openmp's parallel block call a function and returned from it that worked. Why am I not allowed to break/return in between? Also, as an aside, what are the issues/disadvantages of using OpenMP? — quickdraw, Feb 22 '18 at 17:06
@user7795930: it's really better to search online for OpenMP discussions as comments are not suitable for this. Re returning from `parallel for`: I suppose it's not possible because it's **parallel** `for`, so loop condition is not checked on each iteration and later iteration can be already running. Check this [answer](https://stackoverflow.com/a/9813239/453271) — Andriy Tylychko, Feb 22 '18 at 17:13

Daniel Langr · Answer 3 · 2018-02-22T12:45:00.503

-1

You can use std::async with default launch policy. However, this is not the same as thread pool.

In OpenMP, you can set a fixed number of threads and then use OpenMP tasks. Unfortunately, there is no such option in C++11. The Standard says that the choice whether the function will be invoked asynchronously in a new thread or synchronously in a thread that calls wait or get on a corresponding std::future object can be deferred, however, then still a new thread must be created when asynchronous invocation is selected.

edited Feb 22 '18 at 12:45

answered Feb 22 '18 at 11:46

Daniel Langr

22,196
3
50
93

1

This does not really answer the question. Op is not using OpenMP, and as you pointed out yourself, `std::async` is not a viable solution for this problem (at least not until we get executors in C++). – ComicSansMS Feb 22 '18 at 14:05
@ComicSansMS Agree, it should be a comment. – Daniel Langr Feb 23 '18 at 07:36

Threadpool - CPU usage?

3 Answers3