How do I prevent Java's OutOfMemoryError from occurring on file load?

Question

I am working on a program that loads files and uses their information to populate a Model class. These files can range in size from a few KB to almost a GB. When their sizes fluctuate, so too does the memory used by the Model class.

When I load a file that is large enough, I receive the OutOfMemoryError.

I am able to reject a users request to load a file (based on the file size). How do I examine a file, determine its size, and then determine if the application can handle it?

Is there a way to make this adapt to the amount of RAM on a users' computer?

Just a though. Based on `RAM` you have you can partition the file and load it into memory in chunks and populate the `model` class. — Prateek, Sep 26 '13 at 21:10
Try looking at this question: [Using java to get os level system information][1] [1]: http://stackoverflow.com/questions/25552/using-java-to-get-os-level-system-information — BobbyD17, Sep 26 '13 at 21:11
@Prateek As much as I love the idea of that, I don't think it's the file that's overloading the heap. I think the `Model` is growing too big - so splitting it up wouldn't make too much a difference. — sdasdadas, Sep 26 '13 at 21:13

lreeder · Answer 1 · 2013-09-26T21:13:59.480

2

Use File.length() to determine the size of the file, and reject if it's too big.

How do you know if it's too big? Use Runtime.freeMemory() to see how much heap you have left. Of course loading the file and populating it into your Model class will take additional memory, and you may not actually keep the entire content of the file. You should leave adequate room to load all the data in the classes based on what you know about how you will use the data.

Note that a JVM is not typically allocated all the memory in a users computer, but probably defaults to some smaller number. It can be modified with the Xmx option if the user is not happy with the default.

edited Sep 26 '13 at 21:13

answered Sep 26 '13 at 21:07

lreeder

12,047
2
56
65

Yes, but how do I know if it's too big? What might be too big for one users' computer is different for another users' computer. – sdasdadas Sep 26 '13 at 21:08
Determine and set an upper limit that you know will still be supported on older machines, so you'll never run out of memory. The newer machines will never make use of all of the memory, but the older machines can still use your site. – Gary Hayes Sep 26 '13 at 21:15

score 2 · Accepted Answer · answered Sep 26 '13 at 21:10

You can query the JVM to see how much free and used memory there is :

/* Total amount of free memory available to the JVM */
System.out.println("Free memory (bytes): " + 
Runtime.getRuntime().freeMemory());

/* This will return Long.MAX_VALUE if there is no preset limit */
long maxMemory = Runtime.getRuntime().maxMemory();
 /* Maximum amount of memory the JVM will attempt to use */
System.out.println("Maximum memory (bytes): " + 
(maxMemory == Long.MAX_VALUE ? "no limit" : maxMemory));

/* Total memory currently in use by the JVM */
System.out.println("Total memory (bytes): " + 
Runtime.getRuntime().totalMemory());

Thanks, the `freeMemory()` command is what I'm looking for and you beat lreeder to the punch with that. I'll accept when the time is up. — sdasdadas, Sep 26 '13 at 21:13

How do I prevent Java's OutOfMemoryError from occurring on file load?

2 Answers2