1

I am running the following code using the java -Xmx60g command.

Each array should be around 8.5GB, for a total of 17GB. The machine has 64GB total, with 63GB "free." It prints DONE DECLARING RIVER HANDS 1, indiciating that it finished declaring the first array.

But I get Exception in thread "main" java.lang.OutOfMemoryError: Java heap space during the declaration of the second array. I am confused, because my understanding is that the -Xmx60g should allocate 60GB, while the arrays only use 17GB.

Thanks in advance for your help! 

long NUM_RIVER_HANDS = 2428287420L;
int half_river = (int)(NUM_RIVER_HANDS/2);
byte[][] river_hands_1 = new byte[half_river][7];
System.out.println("DONE DECLARING RIVER HANDS 1");
byte[][] river_hands_2 = new byte[(int)(NUM_RIVER_HANDS - half_river)][7];
System.out.println("DONE DECLARING RIVER HANDS 2");
o0'.
  • 11,739
  • 19
  • 60
  • 87
beserious
  • 101
  • 1
  • 8
  • Are you using a 64-bit JVM? See this post: http://stackoverflow.com/questions/1434779/maximum-java-heap-size-of-a-32-bit-jvm-on-a-64-bit-os – Eric Galluzzo Feb 20 '13 at 14:55
  • Have you checked the resource monitor of you operating system? It could be that your assumption about the amount of memory things 'should' take is off. – akaIDIOT Feb 20 '13 at 14:55
  • What does Runtime.maxMemory() return? What does Runtime.freeMemory() return before and after you've allocated your first array? Have you tried running a profiler against your code? – Mikkel Løkke Feb 20 '13 at 15:09

1 Answers1

4

In the first allocation, you're creating 1214143710 arrays, each of which is 7 bytes plus the object overhead. If we assume a per-object overhead for an array of 16 bytes (which is reasonable, if not conservative) then that means two thirds of your space is being wasted. If we assume 24 bytes for each array in total, that's ~29GB just for the first allocation... and if there's more overhead than that, pushing it to 32 bytes for each array, for example, it's ~38GB, at which point it's not surprising that you can't do it twice :)

Just change the way you're using the arrays to make this much more efficient:

// Names changed to follow Java naming conventions
byte[][] riverHands1 = new byte[7][halfRiver];
byte[][] riverHands2 = new byte[7][(int)(NUM_RIVER_HANDS - halfRiver)];

You'll need to change how you use the arrays as well, but now you have a total of 14 byte[] objects, each of which is very large... but you'll have hardly any wasted memory due to object overheads.

Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Thanks a lot Jon Skeet, that fixed it. Thanks also for informing me of the Java naming convention. – beserious Feb 20 '13 at 18:03
  • Whereas this is a great advice, I am finding it hard to explain what I am seeing. When I declare `long NUM_RIVER_HANDS = 1L << 33;` which is much bigger than `2428287420L`, the allocation goes fine and I see both the println's with default settings of heap memory on both Mac/Linux with 1.8.0_131. The moment I switch to declaring with `2428287420L`, I get `java.lang.OutOfMemoryError: Java heap space`. Does this involve some word boundary issues? Adding @apangin for some advice here. – Kedar Mhaswade Dec 21 '17 at 18:26
  • @KedarMhaswade `(int)((1L << 33)/2)` is 0. Higher bits are truncated. Obviously, there is enough memory to create two zero-sized arrays. – apangin Dec 21 '17 at 21:01
  • Grr. Silly me. Thank you @apangin! – Kedar Mhaswade Dec 21 '17 at 21:47