4

I just started to work with threads and quickly I came to the very common question of how many threads are too many?
After doing some research I got even further confused.

task
I have 16 cores and a application generating 1000 objects, being despite from construction parameters consistent. Creating 1000 threads would usually create a huge overhead destroying the performance advantage from multithreading according to msdn documentation and Maximum number of threads. Also I must stay within the 3.5 NET 32 bit limit (Maximum number of threads in a .NET app?).

Question 1
Having such a unilateral task, is it possible to create more threads than cores on the computer? Is .NET optimizing something in there? What is the maximum number of threads I should use?

Question 2
Is there a simple beautiful solution, other then queueing it with limit resources? Something like a semaphore where I create all the 1000 threads but immediately inactivating the threads in order to not allocate the default thread stack?

Community
  • 1
  • 1
Nex
  • 421
  • 1
  • 4
  • 17
  • 3
    Thread Pool is a helpful concept. – Chris O Jul 09 '15 at 18:38
  • 4
    You should not create that many threads, create as many threads as the number of cores and reuse the thread to complete more than one task. – Marko Jul 09 '15 at 18:40
  • 4
    What is missing from your question is what you think a thread does. Why do you think you need 1000 threads to create 1000 objects? – CodeCaster Jul 09 '15 at 18:42
  • First question have good existing duplicate. Second question (should not be asked in the same post really...) is very broad at this point and unlikely to get concrete answer - in any case please consider submitting it as separate question with link to this one and clarifications on what exactly you hoping to optimize with multiple threads. – Alexei Levenkov Jul 09 '15 at 19:19

3 Answers3

10

Yes, you can (and likely should) create more threads than there are cores, because the operating system scheduler will interupt to swap among active threads, even if nothing else is waiting on that core. However, you also don't want to create 1000 separate threads. Instead, create a number of threads and divide the work among them, so that each thread handles more than one item from the full job.

I've found that a good rule of thumb is to go for two threads per logical core (this counts hyper-threaded cores... if you have an 8-core cpu with hyperthreading for 16 logical cores, create 32 threads). The idea is you want as few scheduler interupts/context swaps as possible, but at the same time keep all of the cores busy working mainly on your task. Given scheduler interupts will still occur, even if nothing else is active on the logical core, having two active threads for that core means it's likely for the scheduler to just put in the idle thread from your program. Even if other stuff is active for that core, it's now still likely to be your thread chosen for execution. Going higher than this can encourage more context switches than is necessary and hurt performance.

The short version is the cost of a second thread per core is low (because the context switches still happen), but the payoff is potentially high (entire scheduler blocks where the cpu is working on your app instead of something else). As you add more threads per core, you start to increase the costs and decrease the potential benefit.

But that's just my (very limited) experience. It's extremely generalized, and not good for much more than a starting point. You really need to profile how your app behaves with different thread numbers to get an idea of how to tune this for best performance.

Finally, in the .Net world it's worth mentioning the ThreadPool and async Tasks. This isn't the best place to get into a full tutorial on those subjects, but reading up on them is well worth your time.

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794
4

is it possible to create more threads than cores on the computer?

Yes, this is possible.

Is .NET optimizing something in there? What is the maximum number of threads I should use?

Actually, you shouldn't care about number of threads in 99.99% of cases, because you... must not create threads manually in 99.99% of cases. Use ThreadPool instead, if you must stay within .NET 3.5. Use tasks, if your version is 4.0 or higher.

Is there a simple beautiful solution, other then queueing it with limit resources?

In fact, no. You can't jump over physically available resources, this is the bottle-neck.

Dennis
  • 37,026
  • 10
  • 82
  • 150
2

Basically you get less and less percentage of performance out of new threads as you make more. 10 threads isn't running 10 times faster than 1 thread. Also, when you get more threads running than there are cores you don't get anything meaningfull from it.

Rule of thumb is to use threads for performance heavy code to let it run in the background instead of on the UI-thread such that you avoid blocking the UI responsiveness. If you want to optimize code that are already non-UI-block, then you can look at the number of cores on the target-machine and create some more threads to help heavy algorithms if it makes sense. Remember that it is much harder to debug and much harder to maintain and write multi-threaded code.

Therefore, only do it to experiment and learn and otherwise only when you really need to squish some extra power into the application.

Answer 1: Yes it is possible to create more threads than cores, but it makes next to zero sense.

Answer 2: Don't create 1000 threads to create 1000 objects! that's not beautiful

Zillo
  • 321
  • 2
  • 8