2

I have a problem with the Garbage Collector of my application, I am working in a GCP environment with Kubernetes and BigTable as a data source.

Upon receiving loads, the application does not free up memory space and fills up until k8s restarts the POD. I use a profiler to see the behavior of the JVM and this is the result.

In the "Old Gen" pool it is seen that the memory is full but it never frees the space and it fills up until restarting and starting again.

Old Gen

In the pool "Eden Space" it is seen that while it is filling, the space is freed and never reaches the limit.

Eden Space

This is the JVM configuration when creating the docker image to deploy it in k8s.

java
-XX:+UnlockExperimentalVMOptions
-XX:+UseG1GC
-XX:+UseCGroupMemoryLimitForHeap
-XX:+AlwaysPreTouch
-XX:ParallelGCThreads=5
-XX:GCTimeRatio=4
-XX:MaxGCPauseMillis=100
-XX:MinHeapFreeRatio=30
-XX:MaxHeapFreeRatio=30
-Xms512m
-Xmx4608m

This same configuration is applied in another application that makes REST calls and has never had memory filling problems as in this case.

Java version is 1.8

Thankful for your help.

Grettings.

Elias Vargas
  • 1,126
  • 1
  • 18
  • 32
  • 1
    Could you please try to enable extra logging by -XX:+PrintGCDetails and -XX:+PrintGCTimeStamps to see GC behaviour? Based on your graph, I can conclude that "In the "Old Gen" pool it is seen that the memory is full but it never frees the space and it fills up until restarting and starting again.". – Alexander Makarov Sep 07 '20 at 02:02

1 Answers1

2

Firstly, you have to identify the root cause of your problem to proceed further. I would suggest you to collect more information on the topic with -XX:+PrintGCDetails and -XX:+PrintGCTimeStamps. If your hypothesis that "Old Gen" pool is never frees the space is correct (honestly spearking, it's not obvious from the image), you might apply few tricks to deal with it as such:

  • Directly specify new and old gc sizes via -XX:MaxNewSize, -XX:NewSize, -XX:MaxOldSize, -XX:MaxOldSize to ensure that there is enough space for old generation
  • Gather more logs via -XX:+PrintGCDetails and -XX:+PrintGCTimeStamps to see if any issue happens due to over utilizing of one of the generations
  • Try to increase survivor space ratio via -XX:SurvivorRatio to expedite collection of the old generation objects
Alexander Makarov
  • 876
  • 5
  • 14
  • Thanks for your answer, as you mentioned the problem was with some objects ExecutorService in multiple threads that were not closing correctly and now it works fine. – Elias Vargas Sep 09 '20 at 13:31