1

Has anyone had problems with the JVM (Hotspot) leaking memory when the G1 collector is used?

I've fixed the heap size to 60GB (both -ms and -ms are set to 60G), but the size of the java process (according to the vsz column of the ps command) starts off at around 64GB, but increases to 84GB within 7 hours.

Using the parallel collector, the process size remains steady over a 20 hour run, at around 65GB or so.

Has anyone else had similar problems with the G1 collector? I'm running a very simple benchmark, and I'm not using any direct buffer memory, or other off-heap memory (that I'm aware of).

Java version is 1.7.0, update 5

(I've raised a bug with Oracle about this, but thought I'd check here as well in case anyone has a workaround).

Neil
  • 1,754
  • 2
  • 17
  • 30
  • 1
    Send your question to hotspot-gc-dev@openjdk.java.net . – Hot Licks Jun 17 '12 at 20:37
  • @Neil Can you tell us wich Java version you're using? – alain.janinm Jun 19 '12 at 10:41
  • @alain.janinm Sure, it's 1.7.0, update 5. I've updated the question. – Neil Jun 19 '12 at 16:09
  • there is no directbuffer involved in the test(btw, there is a way to get the amount of memory used by them too), no finalization either, the LinkedList has trivial size, so no issues there. A few programming issues with the test itself: for "bounded" LHM you can use the protected method `removeEldestEntry` but that has no bearing on the test, there is a data race and unneeded sync. processing the stats. Yet, all that doesn't affect the test at all. – bestsss Jun 19 '12 at 17:21

1 Answers1

1

Has anyone else had similar problems with the G1 collector?

Shortly - yes.

here is SO topic about causing memory leaks:

Creating a memory leak with Java

it contains info about G1

Using InflaterInputStream passing new java.util.zip.Inflater() in the c-tor (PNGImageDecoder for instance) and not calling end() of the inflater. Well, if you pass in the c-tor w/ just new, no chance... and yes calling close() on the stream does not close the inflater if it's manually passed as c-tor parameter. This is not a true leak since it'd be released by the finalizer... when it deems it necessary. Till that moment it eats native memory so badly it can cause linux oom_killer to kill the process with impunity. The main issue is that finalization in java is very unreliable and G1 made it worse till 7.0.2. Moral of the story: release native resources as soon as you can, the finalizer is just too poor.

leak is also mentioned here: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7152954

Community
  • 1
  • 1
dantuch
  • 9,123
  • 6
  • 45
  • 68
  • Thanks very much for the info. I'm not using inflaters, and I'm not calling System.gc() explicitly (and I'm pretty sure nothing I'm using is either), so I don't think the problems I'm seeing are related to either of those. – Neil Jun 17 '12 at 20:44
  • @Neil it's really hard to tell what's wrong. Memory leaks are pain in ass, alwalys :) – dantuch Jun 17 '12 at 20:45
  • @Neil, i am the one who posted the issue w/ G1, which version do you use? And most importantly, does it leak native memory? direct ByteBuffers are similar to Inflater/Deflater (they use native C memory/malloc). System.gc() is not bad overall. If it's not native memory, jmap -histo should be enough to trace the issue. Also if you can post the benchmark it'd be useful. – bestsss Jun 17 '12 at 22:06
  • Yes, it appears to be leaking native memory -- it's certainly not heap memory, since I have around 10GB free on the heap throughout my test (after GC has happened). – Neil Jun 18 '12 at 08:53
  • I'm not using direct byte buffers directly (and my test is very simple, so I'd be surprised if I'm calling anything that does). Just in case, I've set -XX:MaxDirectMemorySize=1G. Code is here: https://gist.github.com/2947570 – Neil Jun 18 '12 at 09:02