Number of ThreadPoolExecutor threads keep growing

Question

An application of us creates more and more threads. It seems that all these threads are created by ThreadPoolExecutors.

Here is an example of the output of jstack for such threads:

"pool-6912-thread-1" #37152 prio=5 os_prio=0 tid=0x00007f364445c800 nid=0x71e7 waiting on condition [0x00007f3685ad9000]
   java.lang.Thread.State: WAITING (parking)                                                                                                                      
  at sun.misc.Unsafe.park(Native Method)
  - parking to wait for  <0x00000007712f2f00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
  at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
  - None

"pool-6911-thread-1" #37151 prio=5 os_prio=0 tid=0x00007f3644108800 nid=0x71e6 waiting on condition [0x00007f36a0bcc000]
   java.lang.Thread.State: WAITING (parking)
  at sun.misc.Unsafe.park(Native Method)
  - parking to wait for  <0x000000077114f358> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
  at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
  - None

"pool-6910-thread-1" #37142 prio=5 os_prio=0 tid=0x00007f3644493000 nid=0x71a7 waiting on condition [0x00007f36225a4000]
   java.lang.Thread.State: WAITING (parking)
  at sun.misc.Unsafe.park(Native Method)
  - parking to wait for  <0x0000000770a0aa68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
  at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
  at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
  - None

This seems to be quite the same as in question Too many thread pools with few threads in each pool.

However, all our ThreadPools are created with one of

Executors.newSingleThreadScheduledExecutor();
Executors.newFixedThreadPool(1);
Executors.newScheduledThreadPool(1);

and all these ExecutorServices are final fields in classes that are always created only once.

How can we find the source of these threads? Unfortunately the strack trace has no indication about which ExecutorService is causing this problem.

Is it possible that any of these ExecutorServices is causing this?

Can these threads be created by something other that the above mentioned methods of Executors?

Is there any other way to find the cause of these problems?

Try specifying ThreadFactory when creating pools and set thread name in that factory. See https://stackoverflow.com/a/9748697/7417402 on how to do this. This way you'll at least be able to identify if those threads are created by your own code — Devstr, Feb 20 '18 at 10:58
Somehow you have more than 6900 thread pools created (and maybe destroyed) so far. Are you completely sure that those three pools never created more than once? — yegodm, Feb 20 '18 at 11:17
@_Devstr The tip with the ThreadFactory was very good! This way we can see which ExecutorService is the problem. @_yegodm You are right, too! One of those ExecutorServices seems to be recreated every time. We still do not really got the problem, but at least we know which class is causing it! Many thanks to both of you! — radlan, Feb 20 '18 at 15:37

Number of ThreadPoolExecutor threads keep growing

0 Answers0