java does not properly reserve huge initial heap size

Question

I'm working on a single machine with 512GB RAM (addressed by several AMD Opteron 6212 CPUs). There is currently about 300GB RAM free. Running a large java computation by running

java path/to/myApp -Xms280g -Xmx280g > output.txt

should make Java reserve 280GB immediately, and error if that fails. Strangely, no error occurs but top only shows a memory usage of 30.4GB but it doesn't crash. How can this happen? Isn't java supposed to crash if the initial heap size cannot be allocated?

And effectively, I get OutOfMemory/Java heap space/GC overhead limit errors once the 30.4GB are full, well before the 280GB is ever reached. Running with 250GB or 300GB yields a similar 30.3GB ~ 30.4GB limit. I'm running OpenJDK 64-bit server VM with OpenJDK Runtime Environment (IcedTea6) on Gentoo Linux, and there is plenty of free RAM (over 300GB).

Have you checked the behavior and the parameters also with VisualVM. Like if it really allocates it at start time and not during processing some data? — MortalFool, May 25 '14 at 22:17
The program actually starts with only 30ish GB assigned and crashes while the matrix is being built, so the program really starts even though the initialization was not succesful. — user1111929, May 25 '14 at 22:18
What is the result of `java -version`? Also, you may find [this](http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html) useful. — Elliott Frisch, May 26 '14 at 00:30
java version "1.6.0_24" OpenJDK Runtime Environment (IcedTea6 1.11.1) (Gentoo build 1.6.0_24-b24) OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode) — user1111929, May 26 '14 at 00:31
Also, I've checked that page but saw nothing I could use, moreover I feel that this is unexpected behavior anyway, since it assigns less than the -Xms parameter but still starts the program instead of throwing an exception. — user1111929, May 26 '14 at 00:34
I'm assuming that you're running on Linux here, but it could be that while Java is allocating the memory, Linux hasn't given you the memory yet. See: [C program on Linux to exhaust memory](http://stackoverflow.com/questions/1865501/c-program-on-linux-to-exhaust-memory) and [Allocating more memory then there exists](http://stackoverflow.com/questions/19750796/allocating-more-memory-than-there-exists-using-malloc) — rm5248, May 26 '14 at 03:58
"Gentoo Linux" is indeed a linux distribution. However, I don't see how this explains the observed behavior, since the program actually crashes when the 30.3GB is full, instead of allocating the rest, even though 300+GB is available. — user1111929, May 26 '14 at 08:08
I ran a quick test; it would appear as though Java doesn't allocate the entire heap when it starts up. I can allocate a 16GB heap for Java on my computer(8GB memory + 8GB swap), but the amount of free space as shown by `free -m` only drops by 60MB. That doesn't really help, but it could at least partially explain why it's not allocating all of it(using HotSpot 1.7.0_25 on Debian). Quick thought here: Are you sure you can allocate 30+GB of memory?(Check `ulimit -a`) — rm5248, May 27 '14 at 21:53
Have you tried with any different implementation of the JVM? Just to make sure it is not a particular bug of that version of OpenJDK. Emilio's idea of retrieving more data through jconsole is interesting too. — Jorge_B, May 27 '14 at 22:48
@rm5248: The command `ulimit -a` lists max memory size and virtual memory both as unlimited. The command `free -m` shows the same resuls as the ones displayed in `top`. — user1111929, May 28 '14 at 07:50
@Jorge_B: I will ask the sysadmins if it is possible to install another version, but I don't expect that to happen soon. Also, I don't see a comment by an Emilio suggesting anything through jconsole? — user1111929, May 28 '14 at 07:54
@user1111929 Emilio posted the suggestion that `top only show the physical memory, but the kernel could pass some pages to swap partition. use jconsole to check the real jvm memory usage` (since he posted this as an answer and not as a comment it was deleted - that's why you can't see it). — Volo, May 31 '14 at 20:03
you don't need root to install java at all. As long as you can download it, you can run it from ramdisk even. — bestsss, May 31 '14 at 21:56

Diego Giagio · Accepted Answer · 2014-06-02T13:00:08.747

15

The order of the parameters is incorrect. You are passing -Xms280g -Xmx280g as arguments to your own program and not to the JVM. The correct is:

java -Xms280g -Xmx280g path/to/myApp

edited Jun 02 '14 at 13:00

answered Jun 01 '14 at 20:45

Diego Giagio

1,027
7
7

1

I... I can hardly believe it was that simple. But it was. Program running with 280GB assigned memory now. Thanks! – user1111929 Jun 02 '14 at 12:25

score 4 · Answer 2 · answered Mar 25 '15 at 04:07

If you want the memory specified in -Xms to be grabbed during the initialization of your application then use

java -XX:+AlwaysPreTouch -Xms2G -Xmx2G ......

AlwaysPreTouch will demand every memory page during the initialization of the JVM rather than just keeping what is not needed as “virtual”. However, note that this will have some latency in the starting of the JVM.

Use the aforementioned switch and then check with top. You will get full 2G (in fact a little more) for your JVM.

score 0 · Answer 3 · answered May 27 '14 at 22:41

0

Try adding -d64 parameter in the cmdline

java -d64 path/to/myApp -Xms280g -Xmx280g > output.txt

answered May 27 '14 at 22:41

NRJ

1,064
3
15
32

Same result. It's not running in 32-bit mode or so, else it couldn't assign 30 GB. ;) – user1111929 May 28 '14 at 07:43
can you run ulimit -a and paste output – NRJ May 28 '14 at 18:19
can you also mention the linux flavor and kernel version – NRJ May 30 '14 at 19:58
`uname -a` yields `Linux evarist 3.1.6-gentoo #2 SMP Fri Jan 6 18:47:06 CET 2012 x86_64 AMD Opteron(TM) Processor 6212 AuthenticAMD GNU/Linux`, is this what you want? – user1111929 May 31 '14 at 00:00
And `cat /proc/version` yields `Linux version 3.1.6-gentoo (root@cayley) (gcc version 4.5.3 (Gentoo 4.5.3-r1 p1.0, pie-0.4.5) ) #2 SMP Fri Jan 6 18:47:06 CET 2012` – user1111929 May 31 '14 at 00:12
Now try adding these 2 params: – NRJ Jun 01 '14 at 20:30

score 0 · Answer 4 · answered Jun 02 '14 at 06:36

So you are reaching roughly to 32 GB as limit. While Oracle speak about Compressed oops in the Hotspot JVM it also says about 32GB.

On an LP64 system, though, the heap for any given run may have to be around 1.5 times as large as for the corresponding ILP32 system (assuming the run fits both modes). This is due to the expanded size of managed pointers. Memory is pretty cheap, but these days bandwidth and cache is in short supply, so significantly increasing the size of the heap just to get over the 4Gb limit is painful.

(Additionally, on x86 chips, the ILP32 mode provides half the usable registers that the LP64 mode does. SPARC is not affected this way; RISC chips start out with lots of registers and just widen them for LP64 mode.)

Compressed oops represent managed pointers (in many but not all places in the JVM) as 32-bit values which must be scaled by a factor of 8 and added to a 64-bit base address to find the object they refer to. This allows applications to address up to four billion objects (not bytes), or a heap size of up to about 32Gb.

Link here

java does not properly reserve huge initial heap size

4 Answers4