4

Since C++11 there has been a surge in the amount of parallel/concurrent programming tools in C++: threads, async functions, parallel algorithms, coroutines… But what about a popular parallel programming pattern: thread pool?

As far as I can see, nothing in the standard library implements this directly. Threading via std::thread can be used to implement a thread pool, but this requires manual labor. Asynchronous function via std::async can be launched either in a new thread (std::launch::async) or in the calling thread (std::launch::deferred).

I think std::async could've been easily made to support thread pooling: via another launch policy (std::launch::thread_pool) which executes the task in an implicitly created global thread pool; or there could be a std::thread_pool object plus an overload of std::async which takes a thread pool.

Was something like this considered, and if so, why was it rejected? Or is there a standard solution that I am missing?

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
Andrei Matveiakin
  • 1,558
  • 1
  • 13
  • 17
  • Why are you certain `std::async` _doesn't_ use a thread pool? – Useless Sep 15 '21 at 10:53
  • 3
    Either because 1. It hasn't been proposed or 2. Such proposal hasn't been accepted. – eerorika Sep 15 '21 at 10:54
  • I have no idea, think they wanted to keep stl restricted to more primitive building blocks. And no there isn't a standard solution, but I have one lying around. – Pepijn Kramer Sep 15 '21 at 10:55
  • @Useless It is implemenation specific. Only on windows with msvc a threadpool is used, other implementations dont use a threadpool. – Pepijn Kramer Sep 15 '21 at 10:56
  • 2
    it isnt *easy* to get a thread-pool into the standard. I dont know how it came in all details, but consider that C++11 introduced `std::thread` and only C++20 introduced `std::jthread`. I suppose that time was needed to gather experience with the new threading facilities before C++20 could introduce `std::jthread` and this is still low level. I'd expect higher level abstractions to be even less easy to stnadardize – 463035818_is_not_an_ai Sep 15 '21 at 11:00
  • 2
    There are a number of different types of thread pool, and choosing the "best" for some purpose requires pretty detailed knowledge of the application and (quite often) properties of the host system. Those characteristics increase difficulty in getting agreement on what should be standardised. Standardising support of a thread pool that is (say) well-tuned for windows but poorly tuned for other operating systems runs against core philosophies of C++. – Peter Sep 15 '21 at 11:25
  • @Useless, @P Kramer, it was my understanding that implementing `std::launch::async` via a thread pool would be incompliant with the standard, which requires to use [a new thread](https://en.cppreference.com/w/cpp/thread/launch). A thread pool wouldn't have the same behavior: (a) because of thread local variables (see François Andrieux answer); (b) because it can deadlock if there are N active tasks in a thread pool with N threads waiting on some queued async task to finish. @P Kramer, do you know who MSVC implementation remedies these problems? – Andrei Matveiakin Sep 15 '21 at 12:28
  • 1
    @Useless it's invalid to use a threadpool because of `std::future` which requires a *new* thread. In fact MSVC had to patch their `std::async` to *not* use a threadpool for that reason. Specifically http://eel.is/c++draft/futures#state-10 and related bits cause major headaches – Mgetz Sep 15 '21 at 13:47

1 Answers1

6

In principal std::async could use a thread pool and is seems to me that allowing this was the intention. But in practice the existence of thread_local makes it difficult.

From cppreference on std::async with std::launch::async:

[...] execute the callable object f on a new thread of execution (with all thread-locals initialized) as if spawned by std::thread(std::forward<F>(f), std::forward<Args>(args)...) [...]

If the function contains any local thread_local variables, and if std::async used a thread pool, the behavior of running the function would std::async could be different from the behavior of std::thread.

One example may be that the thread_local might not have not have it's initial value the second time its called by the same thread. If you were to use std::thread instead, it would always have the initial value.

Another way the behavior would diverge is thread_local object's destructors would not run in the same way for std::async and std::thread. This is illustrated by Microsoft's attempt at using a thread pool, which I suspect scared off others from trying it. You can read about this non-conformance here : In Visual Studio, thread_local variables' destructor not called when used with std::async, is this a bug?.

To be completely conforming, the implementation would need to "reset" all the thread_local objects anyway. This requires compiler support and starts to look an awful lot like starting a new thread anyway.

François Andrieux
  • 28,148
  • 6
  • 56
  • 87