6

What's the recommended corePoolSize passed to ThreadPoolExecutor/ScheduledThreadPoolExecutor?

Runtime.getRuntime().availableProcessors()? Runtime.getRuntime().availableProcessors() * 2?

From one point, I'd like the CPU (all the cores) to be utilized 100% but as little threads to do it as possible, in order for them to finish as quickly as possible and not have too much penalty on context switching.

On the other hand, some of the threads might not utilize the CPU all the time, like waiting for network. In that case, I'd like new threads to be spawned and keep all the cores busy.

I'm OK if there's a temporary over-usage of the CPU. Better than under-usage and tasks not being handled.

So how can I achieve this thread load balance? Thanks.

AlikElzin-kilaka
  • 34,335
  • 35
  • 194
  • 277
  • I'm trying to find a way for java to dynamically do spawn threads without causing performance issues. For a static solution, I can just set a number. Related: http://stackoverflow.com/questions/1718465/optimal-number-of-threads-per-core – AlikElzin-kilaka Feb 28 '17 at 11:25

4 Answers4

5

Sizing Thread pools depends on the nature of the tasks you are going to execute on that pool. As a general rule, it depends on the ratio between wait time and cpu time and the number of cpus available.

The general formula to apply is:

Sizing Threads Pools

The optimal pool size for keeping the processors at the desired utilization is:

Optimal thread-pool sizing

You can find out more information in Java Concurrency In Practice, section 8.2 Sizing Thread Pools.

Jerome L
  • 607
  • 5
  • 11
  • Sounds like a dynamic formula. How can this be applied to an initialized executor with a predefined `corePoolSize`? – AlikElzin-kilaka Oct 20 '20 at 09:08
  • How would one calculate `W/C`? Does something like `0.1/0.9` make sense? – Islam Azab Sep 05 '22 at 19:58
  • W/C ratio is something you estimate depending on the code you are executing in parallel. For example, W/C for disk operations is usually much higher than 1 since there are I/O operations on disk. W/C is low when executing purely CPU bound code. – Jerome L Sep 08 '22 at 13:55
1

I think it will depend on what you are doing. You already alluded that some threads might be waiting for I/O.

corePoolSize is the number of threads that the Pool will keep around, even if idle. I think you want to set the maximum pool size instead.

Context switching overhead will always be there, and I don't think it will actually increase with more threads since the switching is usually implemented as constant slices (ie. time slices do not get reduced inversely proportional to number of runnable threads).

You'll obviously need at least as many threads as there are cores to achieve full load, how many extra threads you need depends on how quickly the threads finish and how long the I/O takes.

One option is to do the I/O in a separate step with a large pool size so you make sure there is always one that has its I/O step completed already (CompletableFuture's are convenient for this). Then for the next step you use a separate ThreadPool that has a poolsize same as the number of cores that will get fed from the previous step as soon as one thread completes.

Something like this (unverified):

CompletableFuture.supplyAsync(ioTask, ioExecutorWithLargePool)
    .thenAccept(cpuBoundTask, executorWithCpuCoreSizedPool);
Rodia
  • 1,407
  • 8
  • 22
  • 29
john16384
  • 7,800
  • 2
  • 30
  • 44
1

You could look into Amdahl's law to define the number of core. Take your program as if it was a single threaded program and see where you can parallelize . That can give you an idea of N (here we could approximate the number of cores, by the number of threads of the application).

Now in my experience, the best way to size the "best" number of threads remains to create your multi threaded program, and test with as many possible value of thread number and see which one outputs the best performance. (see also Optimal number of threads per core)

Community
  • 1
  • 1
Adonis
  • 4,670
  • 3
  • 37
  • 57
1

As addition to john16384's answer:

Your code Runtime.getRuntime().availableProcessors()? Runtime.getRuntime().availableProcessors() * 2 might be a good idea if your CPU supports two threads per CPU core (multithreading).

cmoetzing
  • 742
  • 3
  • 16