0

Context
I have a metric on a server that publishes the amount of threads that I have at any given time. With a recent deployment, I have noticed the number of threads increase by about 30 threads on average (originally stagnated around 370, now at 400 threads consistently).

What I've done
There are many packages/possibilities that could be the root cause for this increase. This is why I looked into analyzing threads. I learned how to get and got a thread dump but I can't see any useful information for me on why these threads were created/how they are used.

My service is not impacted negatively (latency/CPU/Memory) but I would still like to root cause this issue as it could be grounds for a memory leak.

My Question
If there is some resource to be able to get the class/package that created the thread, that would be very helpful (I have searched online for a while for a resource like that).

Any advice to root cause this is much appreciated!

xingbin
  • 27,410
  • 9
  • 53
  • 103
Akash Kakumani
  • 23
  • 2
  • Threads created by third-party libraries usually have some prefix indicating their source. What are the names of the threads you’re seeing? Is there a pattern? – Abhijit Sarkar Mar 02 '18 at 07:09
  • Thanks for this comment, I just noticed that there is a descriptor of each thread. It was a little verbose so I regarded it as something else initially. – Akash Kakumani Mar 03 '18 at 00:06

1 Answers1

0

Find The Source Of Threads

Use Java Executors instead of dealing with Threads directly if you aren't already. If you are using Executors you can define names for the threads in your thread pool. These names are included in thread dumps of your java process. So if a thread has one of your custom defined names you will know which thread pool created it. It's common practice to maintain different thread pools for different types of tasks in your application and to give the threads in each pool different names. This way the you can determine how many threads each part of your application is creating when you take a thread dump. See this question for how to define custom thread names.

How To Limit Threads

The ThreadPoolExecutor allows you define the maximum number of Threads in the pool. So if you know all the pools you are using and what their maximum sizes are you should be able to define the maximum number concurrent Threads that can run in your application.

You may have to pay special attention to server, client, and networking libraries though since they probably create their own threads. Libraries typically name their threads as well, so you can probably google an unfamiliar name or stack trace in your thread dump to figure out where it's coming from.

ilooner
  • 2,480
  • 15
  • 32