Java char array seems to need more than 2 bytes per char

Question

When I run following program(running with "java -Xmx151M -cp . com.some.package.xmlfun.Main") :

package com.some.package.xmlfun;
public class Main {

    public static void main(String [] args) {
        char [] chars = new char[50 * 1024 * 1024];

    }
}

I need to increase maximum memory to at least 151M (-Xmx151M). Accordingly, when I increase array size the limit needs to be increased:

50 * 1024 * 1024 -> -Xmx151M
100 * 1024 * 1024 -> -Xmx301M
150 * 1024 * 1024 -> -Xmx451M

Why does it looks like java needs 3 bytes per char, instead of 2 bytes as documentation suggests?

Also when I similarly create array of long it seems to need 12 bytes per long, instead of 8, with int it needs 6 bytes instead of 4. Generally it looks like it needs array_size * element_size * 1.5

Compiling with - javac \com\som\package\xmlfun\\*java

Running with - java -Xmx151M -cp . com.some.package.xmlfun.Main

Good question... I don't believe alignment constraints alone can explain that — fge, Jun 27 '13 at 13:43

score 8 · Answer 1 · edited May 23 '17 at 12:11

I guess what you are seeing can be easily explained by how the heap in the JVM is organized.

When you pass the parameter -Xmx to the JVM, you are defining what the maximum heap size should be. However, it is not directly related to the maximum size of an array that you can allocate.

In the JVM, the garbage collector is responsible for allocating memory for objects and for cleaning up dead objects. It is the garbage collector that decides how it organizes the heap.

You usually have something called Eden space, then two survivor spaces and finally the tenured generation. All of these are inside the heap, and the GC divides the maximum heap among them. For more details on these memory pools, check this brilliant answer: https://stackoverflow.com/a/1262474/150339

I don't know what the default values are, and they might indeed depend on your system. I've just checked (using sudo jmap PID) how the memory pools divide the heap in an application I run on a system running Ubuntu 64-bits and Oracle's Java 7. The machine has 1.7GB ram.

In that configuration, I only pass -Xmx to the JVM, and the GC divides the heap as follows:

about 27% for the Eden space
about 3% for each of the survivor spaces
about 67% for the tenured generation.

If you have a similar distribution, it would mean that the largest contiguous block of your 151MB is in the tenured generation, and is of about 100MB. Since an array is a contiguous block of memory, and you simply cannot have an object span multiple memory pools, it explains the behaviour you are seeing.

You could try playing with the garbage collector parameters. Check the garbage collector parameters over here: http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

Your results seem pretty reasonable to me.

Good point, it would be much sounder to test with `Runtime#freeMemory` and `totalMemory`. — Marko Topolnik, Jun 27 '13 at 13:47
However, if the main() is as simple as the OP makes it, that is quite a waste of memory... Nearly half the size of the array itself. I don't believe the different heap regions can explain such an overhead. — fge, Jun 27 '13 at 13:49
@fge, the default configuration (on a 64bit linux box with Oracle's java 7 I use) seems to give 3% of the heap to the "from" space, other 3% to the "to" space. Then it gives about 27% to the "eden" space and finally about the remaining 67% to the "tenured" space. Therefore, if the OP launches his program with a heap of 150MB, the largest contiguous memory region will have about 100MB. That might be called a "waste of memory", depending on how you look at it. — Bruno Reis, Jun 27 '13 at 13:55
Those "details" should make it to your answer, that explains indeed quite a lot, and in fact it pretty much explains the results... — fge, Jun 27 '13 at 13:57

johnchen902 · Accepted Answer · 2013-06-27T14:26:28.433

In Java HotSpot VM, heap is divided into "new generation" and "old generation". The array must be in either of them. The default value of ratio of new/old generation sizes is 2. _{(Which actually denotes old/new=2)}

So with some simple math it can be shown an 151MB heap can have 50.33MB new generation and 100.67MB old generation. Also an 150MB heap has exactly 100MB of old generation. Your array + everything else (such as args) will exhaust the 100MB, hence produce OutOfMemoryError.

I tried to run with

java -Xms150m -Xmx150m -XX:+PrintGCDetails Main > c.txt

And from c.txt

(...)
Heap
 PSYoungGen      total 44800K, used 3072K (addresses...)
  eden space 38400K, 8% used (...)
  from space 6400K, 0% used (...)
  to   space 6400K, 0% used (...)
 ParOldGen       total 102400K, used 217K (...)
  object space 102400K, 0% used (...)
 PSPermGen       total 21248K, used 2411K (...)
  object space 21248K, 11% used (...)

The spaces are not exactly equals to my calculations, but they are near.

http://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html Here says the ratio is 1/2. Not sure this is the right reference, but that is something to consider, and it does not contradict the heart of this answer. — zw324, Jun 27 '13 at 14:12
@ZiyaoWei Yeah, you're right. But the document itself is misleading. — johnchen902, Jun 27 '13 at 14:20

score 1 · Answer 3 · answered Jun 27 '13 at 14:09

If you look at the size of the data (for example with Visual GC), you see that the size of the array is indeed 2 bytes per char.

The problem here is that the JVM tries to fit the whole array in the old generation of the heap, and the size of this generation is constrained by the ratio of new/old generation sizes.

Running with -XX:NewRatio=5 will correct the problem (default value is 2).

darijan · Answer 4 · 2013-06-27T14:15:26.967

I will try to build up on Bruno's answer. I tried this code right now:

public static void main(String[] args) throws IOException {
    char [] chars = new char[50 * 1024 * 1024];
    System.out.println(Runtime.getRuntime().freeMemory());
    System.out.println(Runtime.getRuntime().totalMemory());
    System.out.println(Runtime.getRuntime().maxMemory());
}

And the output was:

38156248
143654912
143654912

It's obvious that 40 MB were left free for some other purposes of the JVM. My best guess would be for the new generation space.

Java char array seems to need more than 2 bytes per char

4 Answers4