0

We are working on a new application which is still in non-prod. We are using java 8. The issue we are facing is the non-heap increases gradually, with in few days to a point that it crashes the linux system.

Our application is running on tomcat and we use a automated deployment method. As part of troubleshooting we did the below test.

XMX = 4gb

  1. Restarted the Tomcat to give it a fresh start. application started with below stats

    • Heap usage = 1.5gb
    • non-heap = 200mb

  2. Without any user testing after tomcat restart, we did a automated deployment, same code just deployed again and noticed the non-heap doubled.

    • Heap usage = 1.5gb
    • non-heap = 400mb

we repeated the deployments multiple times and each time, the non heap increases. Non just the deploy, but just starting/stopping the application(not the tomcat) also increases the non-heap by 200mb.

Basically we narrowed our issue to, when ever we do a deployment the non heap increases. Important thing to note is we don't restart tomcat during the auto deployment. I'm looking for suggestions on should I include the tomcat restart step after each deployment ? or should I include the XX:MaxMetaspaceSize parameter to put a threshold to non heap ?

we have been monitoring our regular heap usage and there are no issues. Garbage collection happens in regular intervals and reclaims the space. But this is not the case with NON-HEAP, even when the GC happens no space is reclaimed from the NON-heap.

trincot
  • 317,000
  • 35
  • 244
  • 286
Raj K
  • 1
  • 1

2 Answers2

0

You should check a potential issue with class loaders, which are loading too many classes and most probably causing this issue.

For example, you can use a tool such as Java Mission Control, just an example, to focus on the highlighted metric below.

enter image description here

If you see the previous number is increasing over the time, you can use this flag:

-verbose:class

With previous flag you will get information about class loading that could be used to identify the cause of the issue.

rcastellcastell
  • 387
  • 1
  • 7
0

should I include the XX:MaxMetaspaceSize parameter to put a threshold to non heap ?

If Metaspace is growing, sooner or later it will hit that max size threshold and your JVM will eventually crash with java.lang.OutOfMemoryError: Metaspace error.

I'm looking for suggestions on should I include the tomcat restart step after each deployment ?

If you can afford it, then I will suggest you to include it for now until you resolve the issue.

Also please follow this SO answer to collect and diagnose the actual problem in your app. Thanks!

suv3ndu
  • 221
  • 5
  • 12