12

I've been thinking about std::async and how one should use it in future compiler implementation. However, right now I'm a bit stuck with something that feels like a design flaw.

The std::async is pretty much implementation dependent, with probably two variants of launch::async, one which launches the task into a new thread and one that uses a thread-pool/task-scheduler.

However, depending one which one of these variants that are used to implement std::async, the usage would vary greatly.

For the "thread-pool" based variant you would be able to launch a lot of small tasks without worrying much about overheads, however, what if one of the tasks blocks at some point?

On the other hand a "launch new thread" variant wouldn't suffer problems with blocking tasks, on the other hand, the overhead of launching and executing tasks would be very high.

thread-pool: +low-overhead, -never ever block

launch new thread: +fine with blocks, -high overhead

So basically depending on the implementation, the way we use std::async would wary very much. If we have a program that works well with one compiler, it might work horribly on another.

Is this by design? Or am I missing something? Would you consider this, as I do, as a big problem?

In the current specification I am missing something like std::oversubscribe(bool) in order to enable implementation in-dependent usage of std::async.

EDIT: As far as I have read, the C++11 standard document does not give any hints in regards to whether tasks sent to std::async may block or not.

ronag
  • 49,529
  • 25
  • 126
  • 221
  • Just as an addition: http://en.cppreference.com/w/cpp/thread/async provides a quite simple and realistic example of blocking `std::async` calls. – KillianDS Feb 20 '12 at 15:50
  • You seem to be assuming that thread pools have a fixed size. In practice, many are dynamically sized, so blocking is not an issue. – Mooing Duck Jan 22 '14 at 23:24
  • @MooingDuck: Neither TBB nor Concrt have "dynamically" sized thread pools. What thread pools do you know of that are dynamically sized? And even if you have a dynamically sized thread pool you instead get the problem of oversubscription and the overhead of whatever heuristics are needed to keep track of when to add new threads and remove old ones. – ronag Jan 22 '14 at 23:31
  • @ronag: Honestly, the last one I used was Java. I don't know when it creates threads, but I know if the "last" thread is unused for 60 seconds, it's removed. For growth, I'd assume that if a new task arrives and threads are in use, queue it and wait 1second. If no threads open up and CPU<80%, create a new thread. That'd be enough I think. – Mooing Duck Jan 22 '14 at 23:34
  • @MooingDuck: Yes, Java and .NET have dynamically sized thread pools using some type of heuristic. I can't really say much about the performance of these. However, my question was related to C++ and the standard, and as far I know, there are no widely used C++ thread pool implementations that uses heuristics (as to why, I don't know). – ronag Jan 22 '14 at 23:36

2 Answers2

11

std::async tasks launched with a policy of std::launch::async run "as if in a new thread", so thread pools are not really supported --- the runtime would have to tear down and recreate all the thread-local variables in between each task execution, which is not straightforward.

This also means that you can expect tasks started with a policy of std::launch::async to run concurrently. There may be a start-up delay, and there will be task-switching if you have more running threads than processors, but they should be running, and not deadlock just because one happens to wait for another.

An implementation may choose to offer an extension that allows your tasks to run in a thread pool, in which case it is up to that implementation to document the semantics.

Anthony Williams
  • 66,628
  • 14
  • 133
  • 155
  • MSVC's async uses a threadpool. I'd imagine when a thread completes a "task", it reports completion, then resets it's thread locals, then waits for a new task. This would mean user code never has to wait for the thread local reset. – Mooing Duck Jan 22 '14 at 23:26
  • I don't know the details of the MSVC implementation, however the standard requires that `thread_local` variables are destroyed before the `future` is made ready. – Anthony Williams Jan 23 '14 at 16:57
  • 1
    While it is true, that thread_local variables have to be re-initialized, the standard states "_as-if_ in a new thread". As-if is the key point here. In detail, it leaves the choice "thread-pool" vs "new thread" to the implementor. Re-initializing thread_locals should be cheaper than creating a new Thread. – Ilendir Jul 31 '14 at 15:22
  • Reinitializing `thread_local` variables can be non-trivial, especially in the face of dynamically-loaded modules. However, in principle you are right. – Anthony Williams Jul 31 '14 at 15:46
1

I would expect implementations to launch new threads, and leave the thread pool to a future version of C++ that standardizes it. Are there any implementations that use a thread pool?

MSVC initally used a thread pool based on their Concurrency Runtime. According to STL Fixes In VS 2015, Part 2 this has been removed. The C++ specification left some room for implementers to do clever things, however I don't think it quite left enough room for this thread pooling implementation. In particular I think the spec still required that thread_local objects would be destroyed and rebuilt, but that thread pooling with ConcRT would not have supported that.

bames53
  • 86,085
  • 15
  • 179
  • 244
  • 1
    If I remember correctly Microsoft announced during the Going Native 2012 conference that `std::async` would use Concrt. I don't know what plans gcc and clang have. – ronag Feb 20 '12 at 15:43
  • MSVC does use a threadpool at this point, neither libc++ nor libstdc++ use pools last I checked. – Mooing Duck Jan 22 '14 at 23:28