2

The recommended size for custom thread pools is number_of_cores + 1 (see here and here). So lets say there's a Spring app on a system with 2 cores and configuration is something like this 

<task:executor id="taskExecutor" 
pool-size="#{T(java.lang.Runtime).getRuntime().availableProcessors() + 1}" />
<task:annotation-driven executor="taskExecutor" />

In this case there will be one ExecutorService shared among several requests. So if 10 requests hit the server, only 3 of them can be simultaneously executed inside ExecutorService. This can create a bottleneck and the results will go worse with larger number of requests (remember: by default tomcat can handle up to 200 simultaneous requests = 200 threads). The app would perform much better without any pooling.

Usually one core can cope with more than one thread at a time. e.g. I created a service, which calls two times https://httpbin.org/delay/2. Each call takes 2 seconds to execute. So if no thread pool is used the service responds in average 4.5 seconds (tested this with 20 simultaneous requests). If thread pool is used the reponses vary depending on the pool size and hardware. I run a test on 4 core machine with different pool sizes. Below are the test results with min, max and average response times in milliseconds

min, max and average response times in milliseconds

From the results, it can be concluded that the best average time was with the max pool size. The average time with 5 (4 cores + 1) threads in the pool is even worse than the result without pooling. So in my opinion, if a request does not take much of a CPU time then it does not make sense to limit a thread pool to number of cores + 1 in web application.

Does anyone find anything wrong with setting a pool size to 20 (or even more) on 2 or 4 core machine for web services that are not CPU demanding?

Community
  • 1
  • 1
Marko Vranjkovic
  • 6,481
  • 7
  • 49
  • 68

1 Answers1

7

Both of the pages you link to explicitly says that this only applies to unblocked, CPU bound tasks.

You have tasks that are blocked waiting on the remote machine, so this advice does not apply.

There is nothing wrong with ignoring advice that doesn't apply to you, so you can and should increase the thread pool size.

Good advice comes with a rationale so you can tell when it becomes bad advice. If you don’t understanding why something should be done, then you’ve fallen into the trap of cargo cult programming, and you’ll keep doing it even when it’s no longer necessary or even becomes deleterious.

Raymond Chen, on his blog "The Old New Thing"

that other guy
  • 116,971
  • 11
  • 170
  • 194
  • In my opinion neither of the links I provided clearly states "unblocked, CPU bound tasks". In web apps (which is majority of apps today) you rarely have code that it's only CPU bound (DB calls, WS calls...), so Amdahl's law can be applied in few situations. Anyway, thanks for clear explanation – Marko Vranjkovic Dec 15 '15 at 08:52