12

I've been battling some memory leaks, and I'm currently baffled by this issue. There's a web application classloader that was supposed to be garbage collected, but it isn't (even after I fixed several leaks). I dumped the heap with jmap and browsed it with jhat, found the classloader and checked the rootset references.

If I exclude weak refs, the list is empty! How is that possible, since an object held only by weak references should get garbage collected? (I performed GC many times in jconsole)

If I include weak refs, I get a list of references, all of which come from one of the following fields:

  • java.lang.reflect.Proxy.loaderToCache
  • java.lang.reflect.Proxy.proxyClasses
  • java.io.ObjectStreamClass$Caches.localDescs
  • java.io.ObjectStreamClass$Caches.reflectors
  • java.lang.ref.Finalizer.unfinalized

I couldn't find any reason why any of those references should prevent garbage collecting the classloader. Is it a gc bug? Special undocumented case? jmap/jhat bug? Or what?

And the weirdest thing... after sitting idle and gc-ing from time to time for about 40 min, without changing anything, it finally decided to unload classes and collect the classloader.

Note:

If you make a claim about delayed collection of classloaders or weak references, then please specify the circumstances in which it happens, and ideally:

  • provide a link to an authoritative article that supports your claim
  • provide a sample program that demonstrates the behavior

If you think the behavior is implementation-dependent, then please focus on what happens in the oracle or icedtea jvm, version 6 or 7 (pick any one of them and be specific).

I'd really like to get to the bottom of this. I actually put some effort into reproducing the issue in a test program, and I failed - the classloader was instantly collected on System.gc() every time unless there was a strong reference to it.

aditsu quit because SE is EVIL
  • 4,019
  • 1
  • 34
  • 48
  • So where's your problem then? – isnot2bad Dec 02 '13 at 20:33
  • Have you tried Java 7 update 40? If it is a bug, it might have been fixed. – Peter Lawrey Dec 02 '13 at 20:37
  • @isnot2bad does all that seem normal to you?! – aditsu quit because SE is EVIL Dec 02 '13 at 20:40
  • @PeterLawrey currently using icedtea 6.1.12.6, I'll try version 7 later – aditsu quit because SE is EVIL Dec 02 '13 at 20:45
  • @aditsu No, I would have expected that the GC cleaned up the unreferenced objects much earlier. On the other hand, finally it did. Maybe there was enough memory available so there was no need for cleanup. It's really hard to argue about GC behaviour without knowing the internals. – isnot2bad Dec 02 '13 at 20:45
  • @isnot2bad the information I found and all my tests (except the above) indicate that a call to System.gc() immediately performs a full garbage collection every time in this vm – aditsu quit because SE is EVIL Dec 02 '13 at 20:50
  • A couple of points to remember: 1) If there is any class that the class loader loaded that is not eligible for collection then the class loader is not eligible for collection. 2) "Class collection" and associated class loader collection is an implementation option and may be implemented to varying degrees of completeness. (And, of course, a class is not eligible for collection if it's referenced from *anywhere* -- object instance or simply another class's constant pool reference.) – Hot Licks Dec 02 '13 at 21:07
  • @HotLicks If anything is holding the classloader, it should show up in the rootset references. And are you suggesting that the classloader gc implementation in icedtea/openjdk is incomplete? – aditsu quit because SE is EVIL Dec 02 '13 at 21:24
  • @aditsu Do you use any special VM options? Which GC is used? – isnot2bad Dec 02 '13 at 21:27
  • @isnot2bad no special options, default gc – aditsu quit because SE is EVIL Dec 02 '13 at 21:30
  • ClassLoader GC is a special case. As I said, ANY reference to it, probably (implementation dependent) including a weak reference, will prevent it from being collected. Likely it was finally collected because GC flushes weak references that are unused for N GC cycles. – Hot Licks Dec 02 '13 at 21:57
  • 2
    @PeterLawrey I just tried with icedtea 7.2.4.3, same thing happens, the only difference is that there's one more weak reference from com.sun.beans.TypeResolver.CACHE. I haven't yet waited 40+ min to see if it eventually collects the classloader. – aditsu quit because SE is EVIL Dec 08 '13 at 16:22

2 Answers2

6

It looks like there's a soft reference involved somewhere. That's the only explanation I could find for the delayed collection (about 40 min). I initially thought soft references were kept until the memory runs out, but I found that that's not the case.

From this page: "softly reachable objects will remain alive for some amount of time after the last time they were referenced. The default value is one second of lifetime per free megabyte in the heap. This value can be adjusted using the -XX:SoftRefLRUPolicyMSPerMB flag"

So I adjusted that flag to 1, and the classloader was collected within seconds!!

I think the soft reference comes from ObjectStreamClass. The question is why jhat doesn't show it in the rootset references. Is it because it's neither strong nor weak? Or because it already found weak references from the same static fields? Or some other reason? Either way, I think this needs to be improved in jhat.

aditsu quit because SE is EVIL
  • 4,019
  • 1
  • 34
  • 48
  • I *suspect* the reason why `jhat` misses it is that `ObjectStreamClass.class` is not reachable via `ClassLoader.classes`, since it is in the bootstrap classpath. In other words, any references held by static fields in JRE classes would be missed. – Jesse Glick Nov 03 '16 at 20:36
2

Classes reside in special memory space - permanent generation. To unload classloader. GC should choose to include perm space into scope of collection. Different GC algorithms have a little different behavior, but generally GC will try to avoid perm space collection.

In my experience, even if classloader is not reachable JVM may end up with OutOfMemoryError before it would try to collect PERM space.

Alexey Ragozin
  • 8,081
  • 1
  • 23
  • 23