Please I got confused about something. What I know is that the maximum number of threads that can run concurrently on a normal CPU of a modern computer ranges from 8 to 16 threads. On the other hand, using GPUs thousands of threads can run concurrently without the scheduler interrupting any thread to schedule another one. On several posts as: Java virtual machine - maximum number of threads https://community.oracle.com/message/10312772 people are stating that they run thousands of java threads concurrently on normal CPUs. How could this be ?? And how can I know the maximum number of threads that can run concurrently so that my code adjusts it self dynamically according to the underlying architecture.
Asked
Active
Viewed 2.7k times
13
-
3In theory you can run as many as you like, but you'll encounter increasing amounts of overhead as the JVM struggles to keep track of the scheduling. – Oliver Charlesworth Feb 08 '14 at 18:32
4 Answers
10
Threads aren't tied to or limited by the number of available processors/cores. The operating system scheduler can switch back and forth between any number of threads on a single CPU. This is the meaning of "preemptive multitasking."
Of course, if you have more threads than cores, not all threads will be executing simultaneously. Some will be on hold, waiting for a time slot.
In practice, the number of threads you can have is limited by the scheduler - but that number is usually very high (thousands or more). It will vary from OS to OS and with individual versions.
As far as how many threads are useful from a performance standpoint, as you said it depends on the number of available processors and on whether the task is IO or CPU bound. Experiment to find the optimal number and make it configurable if possible.
TypeIA
- 16,916
- 1
- 38
- 52
-
4'Of course, if you have more threads than cores' - you mean 'Of course, if you have more READY threads than cores'. – Martin James Feb 08 '14 at 19:13
-
Just for fun: The maximum amount of threads I can create under Windows 7 x64 with the default thread stack size *and* using `STACK_SIZE_PARAM_IS_A_RESERVATION` to make sure that only the minimum memory necessary is actually committed is about 72k with 8gb memory. – Voo Feb 08 '14 at 23:23
5
At any given time, a processor will run the number of threads equal to the number of cores contained. This means that on a uniprocessor system, only one thread (or no thread) is being run at any given moment.
However, processors do not run each thread one after another, rather they switch between multiple threads rapidly to simulate concurrent execution. If this weren't the case let alone create multiple threads, you won't even be able to start multiple applications.
A java thread (compared to processor instructions) is a very high level abstraction of a set of instructions for the CPU to process. When it gets down to the processor level, there is no guarantee which threads will run on which core at any given time. But given that processors rapidly switch between these threads, it is theoretically possible to create an infinite amount of threads albeit at the cost of performance.
If you think about it, a modern computer has thousands of threads running at the same time (combining all applications) while only having 1 ~ 16 (typical case) number of cores. Without this task-switching, nothing would ever get done.
If you are optimizing your application, you should consider the amount of threads you need by the work at hand, and not by the underlying architecture. Performance gains from parallelism should be weighted against increasing overheads of thread execution. Since every machine is different, every runtime environment is different, it is impractical to work out some golden thread count (however, a ballpark estimate may be made by benchmarking and looking at number of cores).
initramfs
- 8,275
- 2
- 36
- 58
-
'This means that on a uniprocessor system, only one thread is being run at any given moment' - more like 'one thread or no thread'. – Martin James Feb 08 '14 at 19:09
5
There is hardware and software concurrency. The 8 to 16 threads refers to the hardware you have - that is one or more CPUs with hardware to execute 8 to 16 threads parallel to each other. The thousands of threads refers to the number of software threads, the scheduler will have to swap them out so every software thread gets its time slice to run on the hardware.
To get the number of hardware threads you can try Runtime.availableProcessors().
josefx
- 15,506
- 6
- 38
- 63
-
The scheduler will do nothing unless the OS is entered by means of a hardware or software interrupt. The 'thousands of threads' are mostly not ready to run and so do not need any 'time slice' to run on the hardware. – Martin James Feb 08 '14 at 19:12
-
@MartinJames unless your OS runs on cooperative threads there will always be a cpu timer triggering the scheduler via hardware interrupt. If the threads are ready to run depends on what they do and having thousands of threads if all they do is sleep is still wasteful (a stack and management structure for each). – josefx Feb 08 '14 at 19:25
-
It's possible to operate a preemptive multitasking OS with no timer interrupt. The other meams of initiating a scheduler run, ie, interrupts from disk, NIC, KB, mouse and software interrupts from I/O requests and inter-thread synchro are hugely important. The timer is just another interrupt source and ignoring the rest is just bad. – Martin James Feb 09 '14 at 10:09
-
-
@MartinJames as long as there are always less threads ready to run than hardware is available you are correct that using a timer is unnecessary, once you get more some processes might starve. On the stack issue: stacks occupy memory, thousands of threads can quickly occupy 2 GB or more and swapping them out only makes it slower to switch back. – josefx Feb 10 '14 at 20:32
0
While all the other answers have explained how you can theoretically have thousands of threads in your application at the cost of memory and other overheads already well explained here. It is however worth noting that the default concurrencyLevel for the data structures provided in the java.util.concurrent package is 16.
You will come across contention issues if you don't account for the same.
Using a significantly higher value than you need can waste space and time, and a significantly lower value can lead to thread contention.
Make sure you have set the appropriate concurrencyLevel in case you are running into issues related to concurrency with a higher number of threads.
Gaurav Agarwal
- 611
- 6
- 18