50

My application currently consumes quite a lot of memory because it is running physics simulations. The issue is that consistently, at the 51st simulation, Java will throw an error usually because of a heap space out of memory (my program eventually runs thousands of simulations).

Is there anyway I can not just increase the heap space but modify my program so that the heap space is cleared after every run so that I can run an arbitrary number of simulations?

Edit: Thanks guys. Turns out the simulator software wasn't clearing the information after every run and I had those runs all stored in an ArrayList.

Lii
  • 11,553
  • 8
  • 64
  • 88
randomafk
  • 783
  • 1
  • 8
  • 13
  • 8
    sounds like a memory leak – Jason S Jul 19 '11 at 14:06
  • 2
    If as you run more and more simulations, your memory foot print is increasing, you have a memory leak. You need to figure out why your memory isn't being reclaimed at the end of a simulation. – mjr Jul 19 '11 at 14:07
  • Yes, Memory leak probably, I don't remember the name, but its an IBM application who let you see visually every thread and stacks. This tool helped us find a memory leak issue with hibernate couple years ago. will try to update you with the name. – Cygnusx1 Jul 19 '11 at 14:09
  • @randomafk: do the simulations depends one to another? Is your program correctly multi-threaded? If it's *"no"* and *"no"* then a very simply way to "fix" your problem would be to execute your simulation from, say, a shell script. You could then only collect the results and even run your simulations on *x* cores even if your Java application is mono-threaded. Besides that the obvious solution would be not to "clear" the heap space but *a)* stop leaking memory and *b)* use a less-memory-hungry approach (for example by using better libraries than the default Java ones, like Trove). – SyntaxT3rr0r Jul 19 '11 at 14:10
  • possible duplicate of [Heap space out of memory in Java](http://stackoverflow.com/questions/7495155/heap-space-out-of-memory-in-java) – Bobby Sep 21 '11 at 06:43
  • Are you getting the standard OOME or are you getting the PermGen variety? You can use JVisualVM to monitor the VM (if you enable JMX port) and then you would know for sure what is going on. – djangofan Mar 15 '13 at 17:38

9 Answers9

38

There is no way to dynamically increase the heap programatically since the heap is allocated when the Java Virtual Machine is started.

However, you can use this command 

java -Xmx1024M YourClass

to set the memory to 1024

or, you can set a min max

java -Xms256m -Xmx1024m YourClassNameHere
om-nom-nom
  • 62,329
  • 13
  • 183
  • 228
Michael Hernandez
  • 381
  • 2
  • 2
20

If you are using a lot of memory and facing memory leaks, then you might want to check if you are using a large number of ArrayLists or HashMaps with many elements each.

An ArrayList is implemented as a dynamic array. The source code from Sun/Oracle shows that when a new element is inserted into a full ArrayList, a new array of 1.5 times the size of the original array is created, and the elements copied over. What this means is that you could be wasting up to 50% of the space in each ArrayList you use, unless you call its trimToSize method. Or better still, if you know the number of elements you are going to insert before hand, then call the constructor with the initial capacity as its argument.

I did not examine the source code for HashMap very carefully, but at a first glance it appears that the array length in each HashMap must be a power of two, making it another implementation of a dynamic array. Note that HashSet is essentially a wrapper around HashMap.

unkx80
  • 201
  • 1
  • 2
13

There are a variety of tools that you can use to help diagnose this problem. The JDK includes JVisualVM that will allow you to attach to your running process and show what objects might be growing out of control. Netbeans has a wrapper around it that works fairly well. Eclipse has the Eclipse Memory Analyzer which is the one I use most often, just seems to handle large dump files a bit better. There's also a command line option, -XX:+HeapDumpOnOutOfMemoryError that will give you a file that is basically a snapshot of your process memory when your program crashed. You can use any of the above mentioned tools to look at it, it can really help a lot when diagnosing these sort of problems.

Depending on how hard the program is working, it may be a simple case of the JVM not knowing when a good time to garbage collect may be, you might also look into the parallel garbage collection options as well.

Michal Bernhard
  • 3,853
  • 4
  • 27
  • 38
mezmo
  • 2,441
  • 19
  • 22
  • 1
    Shouldn't it be -XX:+HeapDumpOnOutOfMemoryError (+ instead of -)? http://www.oracle.com/technetwork/java/javase/clopts-139448.html#gbzrr – Eli Acherkan Jul 19 '11 at 20:03
  • Could be, It was a different, though similar page from the same company that I copied my text from. Thanks for the correction – mezmo Jul 19 '11 at 21:42
  • JVisualVM link also changed to: https://docs.oracle.com/javase/6/docs/technotes/guides/visualvm/intro.html – Mohsen Abasi Nov 26 '17 at 10:35
6

I also faced the same problem.I resolved by doing the build by following steps as.

-->Right click on the project select RunAs ->Run configurations

Select your project as BaseDirectory. In place of goals give eclipse:eclipse install

-->In the second tab give -Xmx1024m as VM arguments.

PSR
  • 39,804
  • 41
  • 111
  • 151
4

I would like to add that this problem is similar to common Java memory leaks.

When the JVM garbage collector is unable to clear the "waste" memory of your Java / Java EE application over time, OutOfMemoryError: Java heap space will be the outcome.

It is important to perform a proper diagnostic first:

  • Enable verbose:gc. This will allow you to understand the memory growing pattern over time.
  • Generate and analyze a JVM Heap Dump. This will allow you to understand your application memory footprint and pinpoint the source of the memory leak(s).
  • You can also use Java profilers and runtime memory leak analyzer such as Plumbr as well to help you with this task.
P-H
  • 377
  • 3
  • 5
2

Try adding -Xmx for more memory ( java -Xmx1024M YourClass ), and don't forget to stop referencing variables you don't need any more (memory leaks).

ivy
  • 5,539
  • 1
  • 34
  • 48
1

Are you keeping references to variables that you no longer need (e.g. data from the previous simulations)? If so, you have a memory leak. You just need to find where that is happening and make sure that you remove the references to the variables when they are no longer needed (this would automatically happen if they go out of scope).

If you actually need all that data from previous simulations in memory, you need to increase the heap size or change your algorithm.

Mario Duarte
  • 3,145
  • 7
  • 27
  • 37
0

Java is supposed to clear the heap space for you when all of the objects are no longer referenced. It won't generally release it back to the OS though, it will keep that memory for it's own internal reuse. Maybe check to see if you have some arrays which are not being cleared or something.

Rocky Pulley
  • 22,531
  • 20
  • 68
  • 106
0

No. The heap is cleared by the garbage collector whenever it feels like it. You can ask it to run (with System.gc()) but it is not guaranteed to run.

First try increasing the memory by setting -Xmx256m

Bozho
  • 588,226
  • 146
  • 1,060
  • 1,140