1

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Using a 2D array of [100000][100000] and other two arrays of [100000] each. I need these three arrays in the whole program so can't free their memory.

Already tried VM Options -Xmx512m in Netbeans

Please be specific and Step by Step, I am newbie in Java and Netbeans.

Thanks in advance for you help....

Manoj Pathak
  • 209
  • 1
  • 5
  • 15
  • 1
    What the type of the array, try run with -Xmx1200m (max of window xp) or more almost up to your maximum free memory – Roy Shmuli Mar 14 '15 at 22:58
  • That 2D array is going to take 10GB if the elements are bytes, 40GB if they're ints... So you're going to need at least `element_size_in_bytes * 10` GB of RAM... – Emil Laine Mar 14 '15 at 23:02
  • You just need to apply what you learnt in arithmetics: 100000 * 100000 = 10 billion. Asuming these are arrays of bytes, that means you need 10 billion bytes, i.e. 10 GB of memory. Your 512MB won't cut it. – JB Nizet Mar 14 '15 at 23:05
  • or 1.2GB for boolean – Roy Shmuli Mar 14 '15 at 23:06
  • 2
    @RoyShmuli That's not guaranteed, booleans can be bigger than 1 bit. – Emil Laine Mar 14 '15 at 23:08
  • @zenith in java primitive boolean take 1 bit, the object that contain it (array) take 12 bytes(depent 32/64 bit) + 100K/8 and the result rounded to full bytes.. for example size of array[7] equal to size of array[1] (booleans) – Roy Shmuli Mar 14 '15 at 23:13
  • 1
    @RoyShmuli, although VM dependent, Java's boolean take a about a bute of memory in practice: http://stackoverflow.com/questions/383551/what-is-the-size-of-a-boolean-variable-in-java – Bart Kiers Mar 14 '15 at 23:23

1 Answers1

6

Let's do some math. You're allocating a 10,000,000,000 element two dimensional array, plus another two arrays of 100,000 elements.

That's 10,002,000,000 elements. If each of them is an int, that's 40,008,000,000 bytes. That's 37.26 Giga bytes.

Your -Xmx512m isn't nearly enough, you need something closer to -Xmx60G if these are really ints or -Xmx15G in the best case scenario, in which the elements are bytes (e.g. booleans). But that will probably won't work since you (probably) don't have enough physical memory. To me it sounds like you need some disk backed storage, or a database.

Either re-think what you're doing and how you're doing it, or use a machine with that much physical memory.

Malt
  • 28,965
  • 9
  • 65
  • 105
  • Thanks Bart... Got your point... Didn't thought in that way... how to work around this? I am working on graphs with around 80K vertices and then finding a shortest path between two given vertices. – Manoj Pathak Mar 14 '15 at 23:47
  • @ManojPathak If you can't fit the entire graph in memory, store it on the disk, and read only relevant pieces of data each time. Something like Neo4j (https://en.wikipedia.org/wiki/Neo4j) might be useful. – Malt Mar 14 '15 at 23:53
  • Thanks for your suggestions they are really helpful :-) – Manoj Pathak Mar 15 '15 at 00:49
  • You have probably chosen wrong algorithm. There is a lot of algorithms, that doesn't require NxN matrix. Consider Dijkstra's algorithm if you have a graph with positive costs or Bellman-Ford's algorithm if you don't or A* if you have a good heuristic. Those algorithms are fast and don't require much space. I've been working at graphs with 100 000's of vertices and millions of edges on high-end desktop computer and it worked well. – tomas789 Oct 12 '16 at 13:58