Java: enough free heap to create an object?

Question

I recently came across this in some code - basically someone trying to create a large object, coping when there's not enough heap to create it:

try {
    // try to perform an operation using a huge in-memory array
    byte[] massiveArray = new byte[BIG_NUMBER];
} catch (OutOfMemoryError oome) {
    // perform the operation in some slower but less
    // memory intensive way...
}

This doesn't seem right, since Sun themselves recommend that you shouldn't try to catch Error or its subclasses. We discussed it, and another idea that came up was explicitly checking for free heap:

if (Runtime.getRuntime().freeMemory() > SOME_MEMORY) {
    // quick memory-intensive approach
} else {
    // slower, less demanding approach
}

Again, this seems unsatisfactory - particularly in that picking a value for SOME_MEMORY is difficult to easily relate to the job in question: for some arbitrary large object, how can I estimate how much memory its instantiation might need?

Is there a better way of doing this? Is it even possible in Java, or is any idea of managing memory below the abstraction level of the language itself?

Edit 1: in the first example, it might actually be feasible to estimate the amount of memory a byte[] of a given length might occupy, but is there a more generic way that extends to arbitrary large objects?

Edit 2: as @erickson points out, there are ways to estimate the size of an object once it's created, but (ignoring a statistical approach based on previous object sizes) is there a way of doing so for yet-uncreated objects?

There also seems to be some debate as to whether it's reasonable to catch OutOfMemoryError - anyone know anything conclusive?

score 5 · Accepted Answer · answered Dec 01 '08 at 20:07

freeMemory isn't quite right. You'd also have to add maxMemory()-totalMemory(). e.g. assuming you start up the VM with max-memory=100M, the JVM may at the time of your method call only be using (from the OS) 50M. Of that, let's say 30M is actually in use by the JVM. That means you'll show 20M free (roughly, because we're only talking about the heap here), but if you try to make your larger object, it'll attempt to grab the other 50M its contract allows it to take from the OS before giving up and erroring. So you'd actually (theoretically) have 70M available.

To make this more complicated, the 30M it reports as in use in the above example includes stuff that may be eligible for garbage collection. So you may actually have more memory available, if it hits the ceiling it'll try to run a GC to free more memory.

You can try to get around this bit by manually triggering a System.GC, except that that's not such a terribly good thing to do because

-it's not guaranteed to run immediately

-it will stop everything in its tracks while it runs

Your best bet (assuming you can't easily rewrite your algorithm to deal with smaller memory chunks, or write to a memory-mapped file, or something less memory intensive) might be to do a safe rough estimate of the memory needed and insure that it's available before you run your function.

System.gc() usually doesn't stop everything. It depends on the underlying garbage collector. — erickson, Dec 03 '08 at 23:44

score 2 · Answer 2 · edited Sep 01 '19 at 20:09

I don't believe that there's a reasonable, generic approach to this that could safely be assumed to be 100% reliable. Even the Runtime.freeMemory approach is vulnerable to the fact that you may actually have enough memory after a garbage collection, but you wouldn't know that unless you force a gc. But then there's no foolproof way to force a GC either. :)

Having said that, I suspect if you really did know approximately how much you needed, and did run a System.gc() beforehand, and your running in a simple single-threaded app, you'd have a reasonably decent shot at getting it right with the .freeMemory call.

If any of those constraints fail, though, and you get the OOM error, your back at square one, and therefore are probably no better off than just catching the Error subclass. While there are some risks associated with this (Sun's VM does not make a lot of guarantees about what happens after an OOM... there's some risk of internal state corruption), there are many apps for which just catching it and moving on with life will leave you with no serious harm.

A more interesting question in my mind, however, is why are there cases where you do have enough memory to do this and others where you don't? Perhaps some more analysis of the performance tradeoffs involved is the real answer?

You've hit the real-life issue on the head - the actual fix was to rewrite the method so that it didn't need lots of memory to begin with. The question was more academic, but still interesting... — Dan Vinton, Dec 01 '08 at 20:09
@jsight on "Why are there cases where.." What about you're ( and I'm not saying it should be done in memory ) encrypting a file system file. Sometime the whole file is 1 mb, others is 12 gb. ? There is no way to release enough RAM when that happens!! — OscarRyz, Dec 01 '08 at 21:05
@Oscar: I don't think I'd try to figure out free memory to determine how to deal with those two cases, though. I'd be more likely to base my decision on the file size with the block size determined based on sys memory at startup rather than real-time telemetry. — jsight, Dec 01 '08 at 22:58

score 2 · Answer 3 · edited Sep 01 '19 at 20:12

Definitely catching error is the worst approach. Error happens when there is NOTHING you can do about it. Not even create a log, puff, like "... Houston, we lost the VM".

I didn't quite get the second reason. It was bad because it is hard to relate SOME_MEMORY to the operations? Could you rephrase it for me?

The only alternative I see, is to use the hard disk as the memory ( RAM/ROM as in the old days ) I guess that is what you're pointing in your "else slower, less demanding approach"

Every platform has its limits, java suppport as much as RAM your hardware is willing to give ( well actually you by configuring the VM ) In Sun JVM impl that could be done with the

-Xmx

Option

like

java -Xmx8g some.name.YourMemConsumingApp

For instance

Of course you may end up trying to perform an operation that takes 10 gb of RAM

If that's your case then you should definitely swap to disk.

Additionally, using the strategy pattern could make a nicer code. Although here it looks overkill:

if (isEnoughMemory(SOME_MEMORY)) {
    strategy = new InMemoryStrategy();
} else {
    strategy = new DiskStrategy();
}

strategy.performTheAction();

But it may help if the "else" involves a lot of code and looks bad. Furthermore if somehow you can use a third approach ( like using a cloud for processing ) you can add a third Strategy

...
strategy = new ImaginaryCloudComputingStrategy();
...

:P

EDIT

After getting the problem with the second approach: If there are some times when you don't know how much RAM is going to be consumed but you do know how much you have left, you could use a mixed approach ( RAM when you have enough, ROM[disk] when you don't )

Suppose this theorical problem.

Suppose you receive a file from a stream and don't know how big it is.

Then you perform some operation on that stream ( encrypt it for instance ).

If you use RAM only it would be very fast, but if the file is large enough as to consume all your APP memory, then you have to perform some of the operation in memory and then swap to file and save temporary data there.

The VM will GC when running out of memory, you get more memory and then you perform the other chunk. And this repeat until you have the big stream processed.

while( !isDone() ) {
    if (isMemoryLow()) {
        //Runtime.getRuntime().freeMemory() < SOME_MEMORY + some other validations 
        swapToDisk(); // and make sure resources are GC'able
    }

    byte [] array new byte[PREDEFINED_BUFFER_SIZE];
    process( array );

    process( array );
}

cleanUp();

score 2 · Answer 4 · edited May 23 '17 at 12:26

There are some kludges that you can use to estimate the size of an existing object; you could adapt some of these to predict the size of a yet-to-be created object.

However, in this case, I think it might be best to catch the Error. First of all, asking for the free memory doesn't account for what's available after garbage collection, which will be performed before raising an OOME. And, requesting a garbage collection with System.gc() isn't reliable. It's often explicitly disabled because it can wreck performance, and if it's not disabled… well, it can wreck performance when used unnecessarily.

It is impossible to recover from most errors. However, recoverability is up to the caller, not the callee. In this case, if you have a strategy to recover from an OutOfMemoryError, it is valid to catch it and fall back.

I guess that, in practice, it really comes down to the difference between the "slow" and "fast" way. If the "slow" method is fast enough, I'd stick with that, as it's safer and simpler. And, it seems to me, allowing it to be used as a fall back means that it is "fast enough." Don't let small optimizations derail the reliability of your application.

score 2 · Answer 5 · answered Dec 01 '08 at 20:23

The "try to allocate and handle the error" approach is very dangerous.

What if you barely get your memory? A later OOM exception might occur because you brought things too close to the limits. Almost any library call will allocate memory at least briefly.
During your allocation a different thread may receive an OOM exception while trying to allocate a relatively small object. Even if your allocation is destined to fail.

The only viable approach is your second one, with the corrections noted in other answers. But you have to be sure and leave extra "slop space" in the heap when you decide to use your memory intensive approach.

Java: enough free heap to create an object?

5 Answers5