XPages: Too many instances of Java classes causing java/lang/OutOfMemoryError

Question

We are building xPages applications on Domino 8.5.2 that use custom Java Beans, and from time to time the dev server encounters a java/lang/OutOfMemoryError. It all works fine again after the http task is restarted, but of course this is an absolute killer if it happens on a customer's server. So I had a look at the heapdumps in Eclipse Memory Analysis, and checking for the top consumers I get this:

screenshot 1

Obviously the class BCCPropertyStore is instanciated so often that it makes up 20% of the entire heap, although it is supposed to be singleton. The class begins like this:

public class BCCPropertyStore {

// Constants
private static final String     CLASS_NAME  = "BCCPropertySynch ";
private static BCCPropertyStore instance    = new BCCPropertyStore();

...

public static BCCPropertyStore getInstance() {
    return instance;
}

And every time it is used we call it with BCCPropertyStore.getInstance(), which is static, so my understanding is that there should not be multiple instances of it (the Java beans usually are in application scope, so I don't see why they should have multiple instances either). However, if I look at the duplicate classes it is striking that there also a lot of xPages-classes appearing much more often than they should:

screenshot 2

There are no thousands of users logging into the application for whom all these instances could have been created, just me and a fellow developer. HTTPJVMMaxHeapSize is set to 256M, which in theory should be more than sufficient for an application of the size of this one.

Why is the JVM creating so many useless instances of classes until it runs out of memory, and why are they not snuffed out by the garbage collector? Is this an xPages-specific issue or did I miss something?

UPDATE

Today the error occured again and I am no wiser. I changed BCCPropertyStore into enum as suggested below, obviously that didn't change anything as the heap dump looks pretty much the same as in the screenshots I posted before.

Is there any tool with which we can monitor the memory usage of the JVM while it is running, or something similar which helps us to determine if the fixes and suggestions are working or not?

Here is the stacktrace:

2013-05-07T10:44:32.441+02:00 java.lang.RuntimeException: com.ibm.xsp.FacesExceptionEx: java.lang.OutOfMemoryError at com.ibm.designer.runtime.domino.adapter.ComponentModule.initModule(ComponentModule.java:433) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.initModule(NSFComponentModule.java:427) at com.ibm.domino.xsp.module.nsf.NSFService.loadModule(NSFService.java:561) at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:521) at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:342) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:304) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:261) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:291) Caused by: com.ibm.xsp.FacesExceptionEx: java.lang.OutOfMemoryError at com.ibm.xsp.config.CLBootStrap.initContext(CLBootStrap.java:73) at com.ibm.xsp.config.BootStrap.init(BootStrap.java:60) at com.ibm.xsp.config.ConfigureCoreListener.contextInitialized(ConfigureCoreListener.java:58) at com.ibm.designer.runtime.domino.adapter.ComponentModule.initModule(ComponentModule.java:425) ... 7 more Caused by: java.lang.OutOfMemoryError at java.util.Hashtable.newEntry(Hashtable.java:91) at java.util.Hashtable.put(Hashtable.java:766) at java.util.PropertyPermissionCollection.add(PropertyPermissionCollection.java:40) at java.security.Permissions.add(Permissions.java:98) at org.apache.harmony.security.fortress.PolicyUtils.toPermissionCollection(PolicyUtils.java:541) at org.apache.harmony.security.fortress.DefaultPolicy.getPermissions(DefaultPolicy.java:242) at org.apache.harmony.security.fortress.DefaultPolicy.implies(DefaultPolicy.java:365) at java.security.ProtectionDomain.implies(ProtectionDomain.java:159) at java.security.AccessController.checkPermission(AccessController.java:98) at java.lang.SecurityManager.checkPermission(SecurityManager.java:533) at org.eclipse.osgi.framework.internal.core.Framework.checkAdminPermission(Framework.java:1299) at org.eclipse.osgi.framework.internal.core.BundleHost.getResource(BundleHost.java:266) at com.ibm.domino.xsp.module.nsf.Activator.findResource(Activator.java:84) at com.ibm.domino.xsp.module.nsf.Activator.findResource(Activator.java:103) at com.ibm.domino.xsp.module.nsf.Activator.findResource(Activator.java:103) at com.ibm.domino.xsp.module.nsf.Activator.findResource(Activator.java:67) at com.ibm.domino.xsp.module.nsf.NotesClientClassLoader.getResource(NotesClientClassLoader.java:130) at java.lang.ClassLoader.getResource(ClassLoader.java:438) at com.ibm.domino.xsp.module.nsf.ModuleClassLoader.getResource(ModuleClassLoader.java:117) at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:503) at javax.xml.parsers.SecuritySupport$4.run(Unknown Source) at java.security.AccessController.doPrivileged(AccessController.java:202) at javax.xml.parsers.SecuritySupport.getResourceAsStream(Unknown Source) at javax.xml.parsers.FactoryFinder.findJarServiceProvider(Unknown Source) at javax.xml.parsers.FactoryFinder.find(Unknown Source) at javax.xml.parsers.SAXParserFactory.newInstance(Unknown Source) at org.apache.commons.digester.Digester.getFactory(Digester.java:512) at org.apache.commons.digester.Digester.getParser(Digester.java:686) at org.apache.commons.digester.Digester.getXMLReader(Digester.java:902) at org.apache.commons.digester.Digester.parse(Digester.java:1548) at com.sun.faces.config.ConfigureListener.parse(ConfigureListener.java:1229) at com.sun.faces.config.ConfigureListener.contextInitialized(ConfigureListener.java:328) at com.ibm.xsp.config.CLBootStrap.initContext(CLBootStrap.java:65) ... 10 more 2013-05-07T10:44:33.879+02:00 java.lang.OutOfMemoryError at java.util.HashMap.newElementArray(HashMap.java:282) at java.util.HashMap.rehash(HashMap.java:686) at java.util.HashMap.rehash(HashMap.java:730) at java.util.HashMap.putImpl(HashMap.java:611) at java.util.HashMap.put(HashMap.java:605) at com.ibm.domino.xsp.module.nsf.RuntimeFileSystem.refresh(RuntimeFileSystem.java:269) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.initNSFData(NSFComponentModule.java:565) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doInitModule(NSFComponentModule.java:439) at com.ibm.designer.runtime.domino.adapter.ComponentModule.initModule(ComponentModule.java:412) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.initModule(NSFComponentModule.java:427) at com.ibm.domino.xsp.module.nsf.NSFService.loadModule(NSFService.java:561) at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:439) at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:342) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:304) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:261) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:291) 2013-05-07T10:46:17.582+02:00 java.lang.OutOfMemoryError at java.util.HashMap.newElementArray(HashMap.java:282) at java.util.HashMap.rehash(HashMap.java:686) at java.util.HashMap.rehash(HashMap.java:730) at java.util.HashMap.putImpl(HashMap.java:611) at java.util.HashMap.put(HashMap.java:605) at com.ibm.domino.xsp.module.nsf.RuntimeFileSystem.refresh(RuntimeFileSystem.java:269) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.initNSFData(NSFComponentModule.java:565) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doInitModule(NSFComponentModule.java:439) at com.ibm.designer.runtime.domino.adapter.ComponentModule.initModule(ComponentModule.java:412) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.initModule(NSFComponentModule.java:427) at com.ibm.domino.xsp.module.nsf.NSFService.loadModule(NSFService.java:561) at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:439) at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:342) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:304) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:261) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:291)

> ... which is static, so my understanding is that there should not be multiple instances of it. Singletons do not behave with multiple class loaders http://stackoverflow.com/questions/5352550/meaning-of-java-lang-classcastexception-someclass-incompatible-with-someclass — Frantisek Kossuth, Apr 29 '13 at 11:02
Did you make any progress on this one? Am interested to see if you managed to get the out of memory errors under control — Martin Holland, Jan 23 '14 at 21:29

score 1 · Answer 1 · answered Apr 26 '13 at 13:16

1

This is a long shot but I have heard that many java enterprise servers have problems with memory leaks connected to classloaders, static objects and application redeployment.

Could You possibly check if it is possible to recreate java/lang/OutOfMemoryError through repeating clean/run application cycle?

To make this problem lesser You could try to move your static objects to application scope.

answered Apr 26 '13 at 13:16

W_K

868
4
9

Thanks for all the answers so far. I changed BCCPropertyStore into enum as suggested by Nathan below and cleaned my application a lot since I posted this question. Well, today the OutOfMemoryError occured again - which is much sooner than I expected, so the repeated cleaning seems to be a way to reproduce the situation. We already use the application scope where ever it is applicable. – Sarah Steffen May 07 '13 at 09:06

score 0 · Answer 2 · answered Apr 26 '13 at 12:56

Why is the JVM creating so many useless instances of classes until it runs out of memory?

It must not think they are useless.

why are they not snuffed out by the garbage collector?

Because something still carries a reference to them. Since the class carries it's own static reference, I suspect it's that.

Is this an xPages-specific issue or did I miss something?

The code provided here is inadequate to determine the cause, since we don't see the constructor for the BCCPropertyStore. Also we don't know whether in addition to making this class a Singleton, you have also put it in a scope that could cause it to be serialized.

The very first suggestion I would offer is to switch to the enum pattern for defining a singleton.

public enum BCCPropertyStore {
  INSTANCE;

  public static BCCPropertyStore getInstance() {
    return INSTANCE;
  }
}

score 0 · Answer 3 · answered Apr 26 '13 at 13:29

Is there a reason for not using a managed bean instead of an unmanaged one? If you have a managed bean scoped to application, you can still access it from Java using ExtLibUtil.resolveVariable(). That way it will only create one instance, the JVM (specific to the NSF) will create it when needed and remove it for you when unloaded. So you won't need a static reference to the class inside itself.

If you don't have the Extension Library, looking at the code for that will give you what you need for a static method.

`Application.getInstance().getVariableResolver().resolveVariable("variable", FacesContext.getInstance())` — Tim Tripcony, Apr 26 '13 at 13:31

score 0 · Answer 4 · answered Apr 30 '13 at 20:35

Over and above Nathan's comments above.... You should also use Disk Persistence (ie: Save pages to disk) instead of the in-memory options for heavy-weight custom Java objects - particularly important if high-end scalability is required. In doing so, ensure your bean and it's members are either serializable and/or transient accordingly. This approach will radically reduce heap space usage - disk space is cheaper then RAM too!

XPages: Too many instances of Java classes causing java/lang/OutOfMemoryError

4 Answers4