9

I have been working on a classloader leak in our application and finally got to a point where all references to the CL were gone. In my memory profiling tool (using YourKit and jmap/jhat), I can force a GC that sometimes will immediately get rid of my classloader, but then other times (depends on application usage but that's as specific as I can get) when I force the GC, the CL instance doesn't go away. I capture a memory snapshot and look at the results and it says that this object exists, but is unreachable.

Here's the wacky part... and I only discovered this by accident.

I can force full GC's all I want but the instance just doesn't go away. HOWEVER, after 10-20 minutes, if I do another full GC, it does get collected.

(During this time period the application is mostly inactive (not totally). My "extended" coffee break was the accident that led to this discovery.)

So my concern about a leak here is gone (hopefully), but the question is now more of trying to explain this behavior.

Anyone know what might cause the sun JVM to decide to not collect an unreachable classloader for 20 minutes?

Sun JVM version details:

java version "1.6.0_23"
Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
Scott
  • 888
  • 7
  • 21
  • This might be helpful: http://java.sun.com/developer/technicalArticles/javase/finalization/ – Jeremy Aug 12 '11 at 21:39
  • One more bit of info is probably relevant: I am using the parallel collector here. – Scott Aug 12 '11 at 21:42
  • My experience is quite the same as yours and I could always only explain it the way that a proper GC is a complex thing to accomplish. An interesting 'discussion' about it goes here: http://stackoverflow.com/questions/176745/circular-references-in-java . There are some special cases handled by a Garbage Collector, one being Class Loaders, anothers being weak references. If you happen to prove this by reading some actual GC source code or _proven_ answer, please post it here. Another article discussing GC in Java5 in detail: http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html – Stefan Schubert-Peters Aug 12 '11 at 22:23

4 Answers4

5

I think Jeremy Heiler is on the right track with it being a finalization issue. Basically, even if the GC knows an object is unreachable, it may still be an indefinite amount of time before the object gets actually collected, since it may be enqueued for finalization. The finalizer thread will have to get around to finalizing the object (which could take a while, depending on how many other objects need to be finalized and what their finalizers look like) and then, after the object has been finalized, you'll need another GC to re-discover that the object is unreachable before it actually gets collected.

Plus, in the case of a ClassLoader on a Sun JVM, it's likely allocated directly into PermGen. So you'll need not one, but two full PermGen sweeps (one to get the object into the finalization queue, and then a second to actually collect it). Since PermGen sweeps are expensive, I believe the Sun JVM doesn't do them at all with default settings, and even with various GC tunings, it still is pretty reluctant to sweep PermGen (especially if there isn't a lot of other memory pressure).

If instead of forcing (um, I mean, encouraging) a GC using System.gc(), what if you do something like:

System.gc();
System.runFinalization();
System.gc();

Of course, even this isn't guaranteed to work, but it at least does the minimum necessary work to clean up a large finalization-required object.

Daniel Pryden
  • 59,486
  • 16
  • 97
  • 135
  • Could be right... but I can't prove it given the apparently non-deterministic behavior I'm seeing. In the end, my CL does get collected and I can even see the permgen usage drop, which is good. I just don't like non-deterministic behaviors, especially when it might come back and bite me later from a production issue! :) – Scott Aug 14 '11 at 17:05
  • @Scott: Unfortunately, true garbage collection is always non-deterministic. There's really nothing you can do about that; if you really need deterministic behavior, you should look into [Real-time Java](http://en.wikipedia.org/wiki/Real-time_specification_for_Java). – Daniel Pryden Aug 14 '11 at 23:16
0

A possible reason is that the classloader was held by soft references. Try running with

-XX:SoftRefLRUPolicyMSPerMB=1

and see if that makes a difference. That's what happened in my case.

Community
  • 1
  • 1
aditsu quit because SE is EVIL
  • 4,019
  • 1
  • 34
  • 48
0

Likely because the JVM uses generational garbage collection, but eventually would do a full mark-and-sweep

michel-slm
  • 9,438
  • 3
  • 32
  • 31
  • Unless I'm not understanding the meaning of the terms, seeing the PS MarkSweep major collections counter increment (and the resultant drop in tenured generation heap usage) seems to me to indicate that the full sweep does occur when I asked for it. – Scott Aug 12 '11 at 21:54
0

Just a guess: The Classloader (and the classes it loaded) are not located in the usual heap, but in the permanent generation ("PermGen"). This part of the memory will be swiped much less often than everything else, since in normal conditions, class loaders don't go out of scope.

(There might be some GC configuration option to influence this frequence.) I suppose filling the PermGen will trigger a collection there, too, but you normally don't want to do this.

Why do you need your classloader collected, in fact?

Paŭlo Ebermann
  • 73,284
  • 20
  • 146
  • 210
  • At the risk of oversimplifying our need for GC'in the CL, there are "pluggable modules" that can come and go in the application which contribute code, configuration and other resources. We create and use a special a classloader for these modules to achieve our requirements, but when a module goes, we want to reclaim the memory it consumed, including permgen. – Scott Aug 14 '11 at 16:59
  • I suppose it will go away whenever a new module needs this space ... though I didn't have any experiences with such a dynamic module system. – Paŭlo Ebermann Aug 14 '11 at 17:12