2

Running Tomcat for an Enterprise level App. Been getting "Permgen out of memory" messages.

I am running this on:

Windows 2008 R2 server, Java 1.6_43, Running Tomcat as a service.

No multiple deployments. Service is started, and App runs. Eventually I get Permgen errors.

I can delay the errors by increasing the perm size, however I'd like to actually fix the problem. The vendor is disowning the issue. I don't know if it is a memory leak, as the vender simply say "runs fine with Jrockit". Ofc, that would have been nice to have in the documentation, like 3mos ago. Plus, some posts suggest that Jrockit just expands permspace to fit, up to 4gb if you have the mem (not sure that is accurate...).

Anyway, I see some posts for a potential fix in Java 1.5 with the options "-XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled"

However, these seem to have been deprecated in Java 1.6, and now the only GC that seems to be available is "-XX:+UseG1GC".

The best link I could find, anywhere, is: http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html#G1Options

Does anyone know if the new G1 garbage collector includes the permspace? or am I missing an option or 2 in the new Java6 GC settings that maybe i am not understanding?

Any help appreciated!

  • Are you doing deployments multiple times without restarting tomcat? If so, I think this is due to bug in Tomcat. – kosa Jul 22 '13 at 18:12
  • @Nambari Link to this bug? – nanofarad Jul 22 '13 at 18:13
  • 1
    @hexafraction:There are lot of discussion related to this bug, here is one for example: http://www.coderanch.com/t/590332/Tomcat/Java-lang-memory-exception-Pergem – kosa Jul 22 '13 at 18:15
  • Could be a [classloader leak](http://frankkieviet.blogspot.com/2006/10/classloader-leaks-dreaded-permgen-space.html) – DannyMo Jul 22 '13 at 18:15
  • @Nambari That doesn't look like a bug to me. Having a higher memory footprint isn't a bug, but a memory leak is. Most Tomcat PermGen issues are a direct result of memory leaks (classloader leaks are just one example, thread locals are another) in the application being loaded, not in Tomcat – Colin M Jul 22 '13 at 18:21
  • What value are you setting -XXMaxPermSize to? JRockIt does not have a separate PermGen area in the same way the regular Oracle JVM does, that's why it wouldn't be an issue on JRockit. In fact I think PermGen is going away in Java 8, replaced with something else... – Joshua McKinnon Jul 22 '13 at 18:21
  • @ColinMorelli: Agree. But I think we can still call it as bug, 1) because class unloading is not happening 2) Other containers like Websphere doesn't exhibit similar behavior. – kosa Jul 22 '13 at 18:24
  • -XX:PermSize=256m -XX:MaxPermSize=512m – user2607841 Jul 22 '13 at 18:28
  • @Joshua - I am fairly certain that this is a Classloader leak. I guess I am just trying to find a work around because the vendor won't own it. So I am hoping someone knows the new G1 GC and can offer advice on settings that might be a workaround. Thanks! – user2607841 Jul 22 '13 at 18:31
  • I know this may be a dead end, but does the vendor have release notes with supported configurations, including JVMs? Usually there's a matrix of OS/JVM/AppServer somewhere. Where do they claim their right to deny you support, if they haven't enumerated what is supported? – davidfmatheson Jul 22 '13 at 18:38
  • @davidfmatheson - So, it is interesting you mention this, as their matrix only points to JRE 1.6 for most products, and JRE 1.5 for some products. JRE 1.6 applies to all the apps we use, hence why I chose it. As to their notes and recommendations, they have failed to work up to expectations. Side note, they recommended a -Xms and -Xmx the same size. I thought this was a no no. Thanks! – user2607841 Jul 22 '13 at 18:48
  • RE: "recommended a -Xms and -Xmx the same size. I thought this was a no no." - There's nothing wrong with doing that, and some recommend it as a performance optimization (when performance is more important than memory usage flexibility). It has nothing to do with your permgen issue though. – kaliatech Jul 22 '13 at 19:17
  • Unfortunately if it's a classloader leak, a) No value of MaxPermSize will be sufficient indefinitely, b) No GC option is going to help you. Classes in PermGen can only be GC'd if the classloader itself is GC'able... – Joshua McKinnon Jul 22 '13 at 19:37

2 Answers2

1

I wouldn't just increase the permgen space as this error is usually a sign of something wrong in software/setup. Is their a specific webapp that causes this? Without more info, I can only give basic advice.

1) Use the memory leak detector (Tomcat 6+) called Find Leaks

2) Turn off auto-deployment

3) Move JDBC drivers and logging software to the java classpath instead of tomcat per this blog entry

sevensevens
  • 1,703
  • 16
  • 27
  • I have used the memory leak detector several times, and not found any leaks. When the Application dies, so does Tomcat (even though the service is still "up"). So, at that point it is impossible to check for leaks. I look at auto-deploy, but I am not sure how that is relevent? Drivers are at the java classpath now. -Thanks! – user2607841 Jul 22 '13 at 19:30
0

In earlier versions of Sun Java 1.6, the CMSPermGenSweepingEnabled option is functional only if UseConcMarkSweepGC is also set. See these answers:

I don't know if it's functional in later versions of 1.6 though.

A common cause for these errors/bugs in the past was dynamic class generation, particularly for libraries and frameworks that created dynamic proxies or used aspects. Subtle misuse of Spring and Hibernate (or more specifically cglib and/or aspectj) were common culprits. The underlying issue was that new dynamic classes were getting created on every request, eventually exhausting permgen space. The CMSPermGenSweepingEnabled option was a common workaround/fix. Recent versions of those frameworks no longer have the problem.

Community
  • 1
  • 1
kaliatech
  • 17,579
  • 5
  • 72
  • 84