5

Some Context: We have upgraded the environment of a web application from running on Java 7 to running on Java 8 and Tomcat 8 (64-bit arch, Heap size about 2 GB, PermGen size=256 MB, no constraints on metaspace size). After a while, we started getting the following error:

java.lang.OutOfMemoryError: Compressed class space

which means that the space needed for UseCompressedClassPointers exceeded CompressedClassSpaceSize. At that moment VisualVM showed a 2 GB metaspace size.

Now with the VisualVM tool, we can see the Metaspace size is constatnly increasing with every request about 3 MB, however the heap does not seem to do so. The heap usage has a saw zigzag shape going back to the same low point after every GC.

I can tell that the application is leaking Metadata only when using a Java JAXB operation, but I couldn't prove it with VisualVM.

The application depends on webservices-rt-1.4 as a JAXB implementation provider. The application uses marshalling, unmarshalling. The class generation from XSD is done with maven-jaxb2-plugin-0.13.1.

Update:

After tracing class loading and unloading, I found out that the same JAXB classes is loaded into memory by WebAppClassLoader multiple times but never cleaned up. Moreover, there are no instances to them in the heap. I debugged and I saw that JDK is calls the method javax.xml.bind.JAXBContext com.sun.xml.bind.v2.ContextFactory.createContext by reflection and that's when the class are created.

I though the classes are cleaned up by GC. Is it the responsibility of the classLoader to clean up?

Questions: Is there a way to analyze the metaspace objects? Why do I have a leak in metaspace but not in heap? aren't they related? Is that even possible?

Why would the app work fine with PermGen but not Metaspace?

Bassam
  • 146
  • 2
  • 10
  • 1
    You don’t have a “PermGen size=256 MB”, as Java 8 has no PermGen. But anyway, the data in meta space is associated with classes, so the first thing to look for are `Class` instances. Even if they don’t make up a significant fraction of the heap, they might be the cause for growing meta space, if their number is growing and they are not collected. It seems, [JAXB makes it easy to raise such issues](http://stackoverflow.com/q/3259291/2711488)… – Holger Oct 31 '16 at 17:49
  • take a heap dump, analyze it for leaked classes or classloaders. – the8472 Oct 31 '16 at 20:32
  • @Bassam : did you end up using the singleton option OR did you use the noOptimize flag ? – anjanb Jul 16 '17 at 06:48

2 Answers2

8

I am facing similar issue.

In my case, memory leak was caused by JAXBContext.newInstance(...) invoke.

Solutions:

Lonzak
  • 9,334
  • 5
  • 57
  • 88
maki888
  • 96
  • 3
  • Thanks. I was wondering why the class loader kept loading the classes again and again. I haven't had time to check the JAXB code yet. – Bassam Dec 22 '16 at 16:22
  • "wrap this new instance as singleton (https://java.net/jira/browse/JAXB-581)" Are there any disadvantages to this method ? It appears that the alternative of using "-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true" is a reduced performance. I'm trying to choose between these 2 options. – anjanb Jul 16 '17 at 06:29
  • 1
    I see that https://java.net/jira/browse/JAXB-581 gives error message that java.net is down. I'm assuming that this ticket was for the JAXB project. Anyone knows how to read that page contents ? I looked at the current JAXB github project at https://github.com/javaee/jaxb-v2 -- however, could not find this particular issue, so don't have the tradeoffs of using the singleton options – anjanb Jul 16 '17 at 06:48
  • @Bassam : did you end up using the singleton option OR did you use the noOptimize flag ? – anjanb Jul 16 '17 at 06:48
  • 1
    @anjanb: The singleton was the best choice , It turned out that the leak existed in Java 7 and older, but we didn't notice it because it was grabage collected. – Bassam Jul 19 '17 at 00:26
  • @Bassam : Thank you. Can you share your singleton code ? – anjanb Jul 19 '17 at 03:39
2

I had similar issue and adding -Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true in setenv.sh as JVM OPT arguments it resolve OOM metaspace issue.

harsha v
  • 21
  • 1
  • I am getting Compressed class space for org.jgroups.logging.Log4J2LogImpl.error:Line 95 How can I solve that issue for org.jgroups.logging.Log4J2LogImp – Manish Prajapati Jul 26 '20 at 12:17