0

I've written a correctly working multithreaded code in vs2012. What I'm surprised about, is that I detach nearly 10000 threads, and it works with no issues. Obviously that quantity of threads cannot be handled simultaneously, so some of them are suspended until the others work.

I would like to know if there exists some option in vs2012 to set the maximal quantity of threads allowed to work simultaneously. I need this particularly to not loose too much time on thread switching.
Also I'd like to manage the total amount of memory that my threads are allowed to use.

Thanks in advance.

Grigor Gevorgyan
  • 6,753
  • 4
  • 35
  • 64
  • Do you know how much time your are losing on context-switching? Contex-switch overhead is highly dependent upon the amount of data that needs to be flushed from the L1 cache. The effect of this overhead can be severe if there are many more CPU-intensive threads than cores and these threads use all of the L1 cache space. – Martin James Apr 27 '13 at 13:29
  • @MartinJames, the overhead of all those threads could be mitigated if those threads are designed to spend considerable periods of time blocked on, for example, semaphores (ie not one single semaphore), or I/O. That way they'd not be troubling the scheduler that often. But otherwise yes I agree completely. – bazza Apr 29 '13 at 05:43

3 Answers3

6

It's your job to limit the number of threads you spawn.

As for the memory they use, except from the stack (I'll get back to that just below) it is totally indistinguishable from the rest of your program. Actually that's the whole point of using threads instead of multiple processes (fork in Unix parlance): they all share the same memory. Since any allocation made by a thread is seen by the OS as coming from your process (and not from a specific thread) you can't artificially limit the memory usage of a single thread.

Concerning the stack memory which is allocated to each thread: you may want to read this answer of mine about this topic (especially the first point), hopefully you will realize it is pointless to try to reduce the stack size of your threads.

So, to sum it up, threads are like any other part of your program: you can't put a hard limit either on their number or on the quantity of memory they are allowed to allocate (within the bounds of what your machine can handle, of course). Again, it's your job as a programmer to manage the resources correctly, not the compiler's job, not the OS's job.

You want less than 10000 threads? Then just don't spawn them in the first place. You want them to use less memory? Then don't allocate that memory in the first place. All you have to do is put the necessary safeguards in your code.

Community
  • 1
  • 1
syam
  • 14,701
  • 3
  • 41
  • 65
3

If you don't want more threads than hardware threads, only spawn at max std::thread::hardware_concurrency threads.

Though, in some cases over-subscription might be fitting if your work sometimes blocks due to I/O or other stuff.

Concerning memory management, this is C++, you have almost full control over how much memory you use and when you release it. The threads will only use the memory which the functions you run on them need (beside some internal overhead).

Stephan Dollberg
  • 32,985
  • 16
  • 81
  • 107
2

The maximum number of threads that can be running during any one single scheduler timeslice is as we all know the same as the number of cores in the machine. So if all your 10,000 threads are ready to run (not blocked on a semaphore or I/O, etc) then only a very few of them will actually be running at any one moment.

If you want to reduce that number you could probably exploit core affinity to restrict the set of cores the threads are allowed to run on.

To increase that number you would of course need more CPU cores. Ten thousand threads is mainframe territory...

Restricting the cores used is a way in which a program can 'play nicely' on a machine that has got other things to do as well. AFAIK C++11 doesn't have core affinity methods, but does allow you to get the underlying native thread handle. You can then use that when calling native core affinity functions.

bazza
  • 7,580
  • 15
  • 22