38

I have a J2EE application with some interesting behavior ... the heap seems to behave well, growing and shrinking with garbage collections as expected over time. There is no appreciable overall long term heap expansion. However, the metaspace just keeps steadily growing at about 20 Mb per hour until we hit MaxMetaspace and encounter an OOME. I have tried both the parallel and G1 garbage collectors (jdk1.8.0_40).

The application is not getting re-deployed during the execution, so it doesn't seem like it would be the typical classloader leak. Does anyone have suggestions as to how to track down the source of this leak?

It Worked Yesterday
  • 381
  • 1
  • 3
  • 5
  • Have you found any answer for this yet? – Ricardo Mogg Aug 11 '15 at 15:56
  • Can you provide more information: JEE server, used libraries. – sibnick Aug 17 '15 at 17:16
  • 1
    This was an attempt to run a legacy J2EE app on JBoss 4.2.3.GA under Java 8. This is not a supported configuration, but the client really wanted to try it. I know there have been major changes to the JBoss classloading since then, so I suspect it was class loader problem. Phillipe's guess below regarding the generation of proxies may have some merit. We eventually decided to "bite the bullet" and port the app to Wildfly 8. – It Worked Yesterday Aug 19 '15 at 03:25
  • 1
    I have also faced similar problem, we use wildfly 10, after several redeploys of app Metaspace oom error is thrown. Increasing max metaspace is just a workaround, not a solution. I didn't find any solution yet. – Dmitry Avgustis Mar 24 '17 at 10:21

2 Answers2

7

The main cause for the java.lang.OutOfMemoryError: Metaspace is:

  • either too many classes or
  • too big classes being loaded to the Metaspace.

If you want to recreate the problem use this code snippet:

public class Metaspace {
static javassist.ClassPool cp = javassist.ClassPool.getDefault();

public static void main(String[] args) throws Exception {
    for (int i = 0; ; i++) { 
        Class c = cp.makeClass("eu.plumbr.demo.Generated" + i).toClass();
    }
  }
}

All those generated class definitions end up consuming Metaspace.

Javaassist in Maven repo.

You can find a lot more about OOME here

Karol Król
  • 3,320
  • 1
  • 34
  • 37
7

Do a heap dump and analyze it with Eclipse MAT. Look at the classes you have loaded. Check if there's something unexpected, especially duplicate classes. It also has a classloader explorer.

Edit: In theory you could also be that you're constantly generating proxies.

Philippe Marschall
  • 4,452
  • 1
  • 34
  • 52
  • 1
    We tried the MAT classloader explorer. Unfortunately the app is busy and complex enough and the leak was slow enough that it was still "needle in a haystack" proposition. This configuration was intended to be an interim solution anyway and we reached the point of diminishing returns, so we gave up. – It Worked Yesterday Aug 19 '15 at 03:32
  • I'm not aware of a WildFly issue. OP never did debug the issue, so it could be anything including his/her application. – Philippe Marschall Aug 29 '17 at 19:04