2

I've recently started trying to use threads with std::thread, and I was wondering if there are any downsides to using threads (specifically, in c++). Are there situations where adding more threads would decrease performance? For example, if I'm creating multiple new threads that are meant to each run a server object listening for incoming data, is it bad (performance-wise) to create a new thread for each one of those server instances? Why/why not?

RedheadRights
  • 23
  • 4
  • 2
    "Is it always good for performance to create a thread?" - No! – Jesper Juhl Jul 17 '20 at 20:15
  • Once case in point: https://stackoverflow.com/q/17348228/10077 – Fred Larson Jul 17 '20 at 20:16
  • 1
    Absolutely not! learn from my mistake. https://stackoverflow.com/questions/42620323/why-is-reading-multiple-files-at-the-same-time-slower-than-reading-sequentially – Tony Tannous Jul 17 '20 at 20:17
  • No, not always good performance to create a thread. Thread creation takes time and space. Thread maintenance, by the operating system, takes time. Many times, a single threaded application can finish before the threads are created, processed, and joined. – Thomas Matthews Jul 17 '20 at 20:20
  • 1
    One project I worked on, we created one thread per IP connection. One dev converted that to using a `select` statement, and one thread would handle multiple connections. We found out the tipping point was one thread handling 50 connections was when making a thread became better than having the one thread manage 51 connections. So a 1:1 ratio was terribly inefficient. Lesson learned: profile, profile, profile, measure, measure, measure. – Eljay Jul 17 '20 at 20:22
  • 1
    @Eljay shame select is not used more. Improves performance massively. – Tony Tannous Jul 17 '20 at 20:24
  • There is always a performance overhead of using threads, since they consume resources (CPU cycles to create, scheduler resources to allocate time to them, memory, etc). They only give benefits if observable effect of those overheads are less than the gains obtained by using them. Total CPU time will always increase unless the logic (algorithm) also changes. Elapsed time *may* increase (if code executed by different threads still needs to be forced to be sequential) or decrease (if large parts of the code can safely be executed in parallel). – Peter Jul 18 '20 at 00:49

2 Answers2

5

It can really hurt performance to create too many threads.

If, for example, you have a thread for each server instance, then you cannot do work for one server instance and then work for another server instance without switching threads. There is a cost associated with switching threads.

Another problem is when two or more threads try to access the same collection of information. This can cause contention which can slow the entire system down. Modern CPUs have multiple cores with limited inter-core resources. Threads fighting over the same objects can saturate those inter-core buses.

You definitely want to avoid designs that can't do "more work" without creating more threads. If you have three servers and each have a thread, that's fine. If you have hundreds of clients and have a pool of ten threads that serve them, that's fine. If you have a few special things to do (like monitor for clock changes) that are best done by their own thread, that's fine too.

But generally, a server with a lot of work to do should apportion that work over a fixed collection of threads. The number of threads in that collection should be based on things like the number of cores the system has, the number of I/Os that can usefully pend at the same time, and the amount of unexpected delays (such as hard page faults) expected.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278
  • Thank you, that was very helpful. And can I just say, wow! You posted this answer within 2 minutes of me posting the question. This is the first time I've asked a question, and I had no idea that responses could be that fast. – RedheadRights Jul 17 '20 at 20:30
3

  1. Creating a thread can be expensive. If you have very small amounts of work to do, it might not be worth it. This article's measurements show that creating a thread may take ~milliseconds.

  2. Threads are an abstraction over CPU cores, and while you can basically create as many threads as you want, the number of available cores is fixed. After a certain point, you will get no additional speedup because the hardware has no more to offer, and you may actually slow things down by introducing more bookkeeping and communication overhead.

  3. Even if you were not limited by hardware concurrency, most workloads are not completely parallelizable, and you'll be limited by the non-parallel parts of your problem. See: https://en.wikipedia.org/wiki/Amdahl%27s_law

jtbandes
  • 115,675
  • 35
  • 233
  • 266
  • Are there any graphs of expected speedup with Amdal's law and the actual speedup? – Tony Tannous Jul 17 '20 at 20:28
  • 1
    Here's one example I found in a quick search: https://hal.inria.fr/hal-02404346v2/document Though I'm sure it depends a lot on how the workload is structured and the environment it's run in. – jtbandes Jul 17 '20 at 20:31