How I can programmatically, from within a JVM, determine the state of the heap?
I've figured out the following code can get information about the various GC collectors via JMX notifications and with the addition of some logic, my application may be able to infer that it is running out of memory.
The JMX information is most likely going to be JVM dependent (OpenJDK, Oracle/Sun, ...) as well as version (7/8/9/...) and maybe even hardware platform dependent. Any solution is most likely is not going to be a 100% solution – I'll be happy with 60%.
Right now it is a work in progress but I hope that this code helps others.
GcJmxListener listener = new GcJmxListener();
// get the JMX beans responsible for the garbage collectors
List<GarbageCollectorMXBean> beans =
ManagementFactory.getGarbageCollectorMXBeans();
for (GarbageCollectorMXBean bean : beans) {
if (bean instanceof NotificationEmitter) {
// add a notification listener on each one of them
((NotificationEmitter) bean).addNotificationListener(
listener, null /* filter */, null /* handback */);
}
}
...
/** Listener that gets called with JMX notifications of GC events */
private static class GcJmxListener implements NotificationListener {
@Override
public void handleNotification(Notification notif, Object handback) {
Object userData = notif.getUserData();
if (userData instanceof CompositeData) {
walkComposite(null /* label */, (CompositeData) userData);
}
}
/** called recursively to interrogate a CompositeData tree */
private void walkComposite(String label, CompositeData comp) {
for (String key : comp.getCompositeType().keySet()) {
String keyLabel = key;
if (label != null) {
keyLabel = label + " -> " + key;
}
Object value = comp.get(key);
if (value instanceof CompositeData) {
// going recursive
walkComposite(keyLabel, (CompositeData) value);
} else if (value instanceof TabularData) {
walkTabular(keyLabel, (TabularData) value);
} else {
/* NOTE: here is where we might add logic to detect problems */
System.out.println(keyLabel + " = " + value);
}
}
}
/** called recursively to handle a TabularData tree */
private void walkTabular(String label, TabularData tab) {
for (Object key : tab.keySet()) {
if (key instanceof List) {
CompositeData value = tab.get(((List<?>) key).toArray());
walkComposite(label + " -> " + key, value);
}
}
}
}
The code is not specific to the Garbage Collector beans and may be useful to interrogate other JMX notifications and composite data trees.
Logic Thoughts
In terms of the logic itself, I think that some of the following logic may help to flag a imminent OutOfMemoryError:
- Increased frequency of mark-sweep GC events compared to scavenge. A healthy JVM typically has many more scavenge events but as it runs out of memory, it has to do more mark-sweeps to reclaim memory. The GC algorithms may be named differently depending on the JVM/version.
- The "Old Gen" (or "Tenured Gen") memory pool still very close to being "full" after a mark-sweep event. I'm not 100% sure if I need to determine the size of the pool or if the max information in the notification is enough here. The pool may be named differently depending on the JVM/version.
- Maybe looking the reclaimed space in all of the memory pools is the best way to to determine if the end is near.
- Did the "Old Gen" memory pool increase in size after a mark-sweep event. This isn't always bad given that some "Survivor" objects will move over to "Old Gen" but there may be some telemetry that will help here.
Obviously a work in progress.
