Java - programmatically reduce application load when runs out of memory

Question

No, really, that's what I'm trying to do. Server is holding onto 1600 users - back end long-running process, not web server - but sometimes the users generate more activity than usual, so it needs to cut its load down, specifically when it runs out of "resources," which pretty much means heap memory. This is a big design question - how to design this?

This might likely involve preventing OOM instead of recovering from them. Ideally

   if(nearlyOutOfMemory()) throw new MyRecoverableOOMException();

might happen.

But that nearlyOutOfMemory() function I don't really know what might be.

Answer 1

Split the server into shards, each holding fewer users but residing in different physical machines.

If you have lots of caches, try to use soft references, which get cleared out when the VM runs out of heap.

In any case, profile, profile, profile first to see where CPU time is consumed and memory is allocated and held onto.

Answer 2

I have actually asked a similar question about handling OOM and it turns out that there's not too many options to recover from it. Basically you can:

1) invoke external shell script (-XX:OnOutOfMemoryError="cmd args;cmd args") which would trigger some action. The problem is that if OOM has happened in some thread which doesn't have a decent recovery strategy, you're doomed.

2) Define a threshold for Old gen which technically isn't OOM but a few steps ahead, say 80% and act if the threshold has been reached. More details here.

Answer 3

You could use Runtime.getRuntime() and the following methods:

• freeMemory()
• totalMemory()
• maxMemory()

But I agree with the other posters, using SoftReference, WeakReference or a WeakHashMap will probably safe you the trouble of manually recovering from that condition.

Answer 4

A throttling, resource regulating servlet filter may be of use too. I did encounter DoSFilter of jetty/eclipse.