3

According to this SO Question: "Why does Windows 10 start extra threads in my program?" and Hans Passant answer: Windows 10 start a thread pool for each and every C++ process on Windows 10 (at least when compiled in c++ in VS2013+).

According to Microsoft documentation "Thread Pools" and "Thread Pool API (2018-05-30)", I cannot find the way to join the default process thread pool.

Can I join the default process thread pool and how? ... or do I have to create a new one?

This is a list of few drawbacks I could see of having more than one thread pool per process:

  • More threads created that could have been avoided
  • More sleeping threads
  • More memory taken for additional threads and the manager itself
  • Less efficent thread management algorithm due to possible concurrency between thread pool.

If I have to create new thread pool instead of joining/using one global thread pool per process, does it remove the advantage of having one and only one thread pool per process? Why can't we verify if there is an already created thread pool and use it directly? Why not just being able to join the main process thead pool? Wouldn't be better to have only one thread pool like in C#?

Side note: I'm working on a math algorithm that calculate long enough to be multi-threaded. Also, it is part of a libary into a 3rd party DLL. Being able to join an already created thread pool would have appears more logical to me than created a new one and perhaps interfere with the customer main process threads and another potential thread pool.

After I got a good answer and great informations from Raymon Chen, I discovered this article that I like to share because it helped me a lot to better understand: Top 20 C++ multithreading mistakes and how to avoid them

Eric Ouellet
  • 10,996
  • 11
  • 84
  • 119
  • 1
    "does it remove the advantage of having one and only one thread pool per process?" - what advantage? –  Apr 23 '19 at 20:23
  • Many thread pools mean more sleeping threads, more memory??? Need more? – Eric Ouellet Apr 23 '19 at 20:25
  • 1
    I don't see why you would think either of those things. Anyway, the process threadpool (if there is one) for doing things like loading DLLs is not likely to be something that MS will give you access to, for obvious reasons. –  Apr 23 '19 at 20:28
  • Sorry I cannot see the obvious reasons? Also having many concurrent thread pools would makes their algorithms to find the best amount of thread a bit silly due to unvoidable race condition between them to calculate the best thread amount. – Eric Ouellet Apr 23 '19 at 20:31
  • If you have access to that pool, you can potentially interfere with what it is intended for by MS. –  Apr 23 '19 at 20:33
  • In C# there is one thread pool create by the framework that everybody could use. Thats sound to me a lot more logic. – Eric Ouellet Apr 23 '19 at 20:34
  • You are right if Microsoft (the loader) give you access before they complete usage of it...But nothing prevent them to give you access after that (if ever possible) or otherwise create a user thread pool for which we can join. – Eric Ouellet Apr 23 '19 at 20:36
  • I think you can creation thread pool using default execution environment (that is use those automatically created threads). *By default, a callback executes in the default thread pool for the process.* - https://learn.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-initializethreadpoolenvironment – user7860670 Apr 23 '19 at 20:49
  • @VTT, thanks I can see that a callback will execute in the default thread pool. That would explain why MS need a thread pool. I can't see how I can attach the that thread pool (if possible) and if not, how I can look for an already created thread pool and attach to it? – Eric Ouellet Apr 23 '19 at 21:10
  • 3
    Just call a thread pool function, and you'll use the default thread pool by default. (Using a nondefault threadpool is quite difficult.) – Raymond Chen Apr 23 '19 at 21:16
  • @RaymondChen, WOW ! If you are sure, then this is the answer I was looking for! Thanks a lots! Can you write an answer and let me 1 day or 2 to ensure about the validity? Do you know if it will use the same threads as MS uses to load the process? – Eric Ouellet Apr 23 '19 at 21:19
  • @RaymondChen, I'm not sure it is so difficult to use another thread pool... just using any library (boost, STL, ...) would (I suspect) probably create a specific library thread pool. But I agree using MS API would probably lead to the same thread pool which would makes sense to me. – Eric Ouellet Apr 23 '19 at 21:26
  • 2
    `CreateThreadPoolWork` by default creates the work in the default thread pool. If you want to use a nondefault thread pool, you have to create a thread pool (`CreateThreadPool`) and when queueing work to it, you have to pass a custom environment (`InitializeThreadpoolEnvironment` + `SetThreadpoolCallbackPool`). BTW, I'm assuming you're talking about the system threadpool, since those are the docs you linked to. – Raymond Chen Apr 23 '19 at 22:44
  • @RaymondChen, I don't know exactly. I talked about the default thread pool as the one that should exists per process. It could be a global windows OS thread pool (under the hood), but I have no idea. It would be nice to have documentation about that :-) ... As far as what I red, I think that we can use TLS on thread from thread pool but having a global OS thread pool would makes sharing code (TLS one) between process possible which would makes a security hole (I think)... – Eric Ouellet Apr 24 '19 at 13:57
  • 1
    Each process comes with a default thread pool. By default, the thread pool functions use the default thread pool. So the way to "join the default process thread pool" is to start using the thread pool functions. (Not sure how TLS fits into the picture, or why you've switched to talking about sharing code between processes.) – Raymond Chen Apr 24 '19 at 14:13
  • @RaymondChen, TLS => It was about the last sentence of you previous post: "...about the system threadpool...". Because you refer about system threadpool, I was a bit confuse if the "default thread pool" is the "system thread pool", or if you were refering to a global OS system thread pool. In the later case, a global thread pool, TLS would create a potential security hole. But It was only an observation./// – Eric Ouellet Apr 24 '19 at 14:24
  • @RaymondChen, If i'm not wrong about 20 years old stuff I learned (thread are kernel objects which belongs to a process (also kernel object) which will be destroyed with the process itself. Then my understanding of global os thread pool is surely wrong and you can forget about TLS. Sorry! My misunderstanding. – Eric Ouellet Apr 24 '19 at 14:24
  • By "system threadpool" I meant "the threadpool created by the system (i.e., the ones you linked to)." Because there are other threadpools, like the boost threadpool, the managed threadpool, IOCP threadpools... – Raymond Chen Apr 24 '19 at 14:40
  • @RaymondChen, I'm currently reading about thread pool and would like to verify an information: in https://learn.microsoft.com/fr-ca/windows/desktop/ProcThread/thread-pools in section "Thread Pool Architecture" it says: "The new thread pool is improved because it provides a single worker thread type (supports both I/O and non-I/O)... ". Do you know if this information is still valid and if so, can I say that IOCP use the same system (or default) thread pool as the API I mentioned and the STL : std::async ? – Eric Ouellet Apr 24 '19 at 15:22
  • 1
    The statement is still correct, but you're misinterpreting it. An IOCP has its own thread pool. The point of the documentation is that the thread pool comes with its own IOCP and you can bind I/O to the IOCP built into the thread pool. `std::async` is not part of Windows. You'll have to look at the `std::async` documentation to see what it does. – Raymond Chen Apr 24 '19 at 16:39
  • @RaymondChen, Thanks Raymond. FYI: I went to the link into you SO profile: https://blogs.msdn.microsoft.com/oldnewthing/. I try 2 links but both are bad (404). Thanks for all informations. – Eric Ouellet Apr 24 '19 at 17:33

1 Answers1

1

std::async uses the default thread pool on Windows, so you might like to use that.

More details here and here.

Paul Sanders
  • 24,133
  • 4
  • 26
  • 48
  • Thanks a lot. I just realised that "std" (stl) implementation could be specific to OS (here: Windows) and could use the operating system parts like the thread pool. Great information. I'm just returning to C++ (a little bit) after almost 20 years of C# and realised that I have a lot to learn. Thank you very much. – Eric Ouellet Apr 24 '19 at 14:04