Garbage collection YGCT and Garbage Collection Time keep rising

Question

I'm having a garbage collection problem where on our system, garbage collection time keeps rising lineairly until it reaches a constant of around 25 seconds.

  S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
  0.00  84.18  76.65  35.47  60.16    441   15.581    16    1.834   17.415
 75.72   0.00  97.32  35.47  60.16    442   15.770    16    1.834   17.604
  0.00  64.69  35.56  35.86  60.16    443   16.318    16    1.834   18.153
100.00   0.00  19.91  35.87  60.16    444   16.381    16    1.834   18.215
  0.00  70.61  40.85  36.82  60.17    445   17.488    16    1.834   19.322
 28.80   0.00  19.61  36.82  60.17    446   17.535    16    1.834   19.369
 34.51   0.00  48.01  36.82  60.18    448   17.561    16    1.834   19.395
  0.00  10.86  20.48  37.11  60.21    453   17.979    16    1.834   19.813
  9.04   0.00  47.39  37.12  60.23    454   18.063    16    1.834   19.898
  0.00  71.26   2.65  37.14  60.23    455   18.173    16    1.834   20.007
 63.64   0.00  90.91  38.04  60.23    456   19.562    16    1.834   21.396
  0.00  76.45  42.70  38.04  60.23    457   19.592    16    1.834   21.426

This is just a snapshot of activity as I executed a little load test.

I reviewed some snapshot, at which I'm not an expert, and did in fact find some memory problems. The obvious ones are now resolved.

The machine has the following settings

JAVA_OPTS="$JAVA_OPTS 
-server 
-Xms704m 
-Xmx704m 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/var/log/tomcat6 
-XX:MaxPermSize=192m 
-XX:+UseConcMarkSweepGC 
-XX:+CMSIncrementalMode 
-XX:+CMSIncrementalPacing 
-XX:+DisableExplicitGC
"

We use Java 6, Tomcat6, Spring Framework, Hibernate, EHCache for caching and use Quartz for various scheduling tasks. This item link actually helped resolve some issues which did lead to MAT reporting potential memory leaks, which is no longer the case.

We played around a lot with all sorts of JVM settings on our test box but in all cases Garbage Collection Time kept growing to unacceptable levels, mainly because of the ever growing YGCT. Initially, I thought the cache was the culprit as we do cache object graphs and don't just use the Hibernate Second Level cache. But the memory use of the cache in MAT didn't seem that excessive at about 18Mbs size.

Is this a memory leak or should I just be throwing more memory at it? If it is a memory leak, how can I best debug that in MAT?

WHY -Xms AND -Xmx Equal ? ideally it shouldn't be equal. – Akhi Jun 27 '12 at 12:37 — Akhi, Jun 27 '12 at 12:37

score 1 · Answer 1 · answered Jun 27 '12 at 13:01

1

To my understanding this is absolutely normal. The YGCT column reports a sum of all YGC actions since the JVM startup. The same goes for FGCT and GCT of course.

Do You notice any performance hit after some time or is this question of pure scientific nature?

answered Jun 27 '12 at 13:01

Tomasz Jędzierowski

969
8
19

No, I'm noticing periodic extreme spikes in processing resulting in timeouts. This usually lasts about 30 seconds. After this, everything is super fast again. I assume this is because of garbage processing. – Marc Jun 27 '12 at 14:17

score 0 · Answer 2 · answered Jun 27 '12 at 13:14

The YGC YGCT FGC FGCT GCT will all increase over time as they are cumulative. WHat you should be interested in is the difference.

Either you are not monitoring fast enough or you are GCing quite often. I would try increasing the amount of memory until it doesn't appear to make much difference. e.g. I usually start at an 8 GB heap with a 6 GB eden space, but you may prefer something different.

Garbage collection YGCT and Garbage Collection Time keep rising

2 Answers2