3

So i created a very large array of byte[] so something like byte[50,000,000][16].. so according to my math thats 800,000,000 bytes which is 0.8GB plus some overhead. To my surprise when to do 

memoryBefore = runtime.totalMemory() - runtime.freeMemory()

its using 1.8GB of memory. when i point a profiler at it i get this

https://www.dropbox.com/s/el9vlav5ylg781z/Screen%20Shot%202015-08-30%20at%202.55.00%20pm.png?dl=0

I can see most byte[] are 24bytes not 16 as expected and i see quite a few much larger byte[] of size 472 or more.. Does anyone know whats going on here?

Thanks for reading

darthShana
  • 367
  • 1
  • 4
  • 16

2 Answers2

4

All objects have an overhead to maintain the object, information like "type" aka Class. See What is the memory consumption of an object in Java?.

Arrays also have a length, and the overhead might be bigger in 64-bit Java.

Since you are allocating 50,000,000 arrays of 16 bytes, you get 50_000_000 * (16 + overhead) + (overhead + 50_000_000 * pointerSize). Second part is the outer array of arrays.

Depending on your requirements, you might be able to improve this in one of two ways:

  1. Flip the indices of you 2-dimensional array to byte[16][50_000_000]. This reduces overhead from 50,000,001 to 17 and reduced outer array size.

  2. Use a single array byte[16 * 50_000_000] and do offset logic yourself. This will keep your 16 bytes contiguous and eliminate all overhead.

Community
  • 1
  • 1
Andreas
  • 154,647
  • 11
  • 152
  • 247
  • @darthShana If this answer is good for you, please accept it by clicking the checkmark, so others can see it. – Andreas Aug 30 '15 at 19:24
1

I can see most byte[] are 24bytes not 16

An object in Java has a couple of words of header information, in addition to the space that holds the object's fields. In the case of an array, there is an additional word to hold the array's length field. The length actually needs 4 bytes ('cos length is an int), but the JVM is most likely aligning to an 8 byte boundary on your platform.

If you are seeing 16 byte arrays occupying 24 bytes, then that accounting of space is most likely including (just) the length word.

(Note that the actual space occupied by object / array headers is JVM and platform specific.)

... I see quite a few much larger byte[] of size 472 or more.

Those are unrelated. If some other part of your code is not creating them explicitly, they are most likely created by Java runtime libraries.

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216