I have a computer with 64Gb of RAM and I have few small Java applications that needed up to 4Gb of RAM.
When Java app running on 32 bit platform it can use only 4GB but it uses direct object addresses. When Java app is running on bigger RAM but less than 32Gb then it uses compressed oops (COOPS) with 8 byte padding which adds some small overhead but anyway it's ok. But when run Java app on 64Gb heap then it uses full 64 bit addresses which consumes twice more memory for storing the addresses.
So my question is: can I run a Java app in 32bit mode so it can't use more than 4Gb but it will use the memory more efficiently. And can I run multiple such apps to use the whole 64Gb address space? Maybe I can use Docker somehow to limit memory for the Java process but I'm not sure if the process will be runned in the simple 32 bit mode because it anyway needs to get access to some memory address higher than 32Gb.
Maybe Docker can virtualilze the 64Gb address space to simple 32 bit addresses.
As I see this solution: we started a Java app and give it a part of heap from 32GB to 36Gb. The specified address space is 4Gb so here is enough just 32bit pointer (4 byte integer) so internally in all fields will be used just regular 32 pointer. But 32 bit pointer can be used for addresses from 0 to 4Gb while the app's heap starts from 32Gb. So internally JVM just converts 32bit pointer to 64 bit by adding 32Gb. This is similar to COOPS but instead of 8 byte padding we alwaysjust add address offset where the app's heap begins. But here it comes another problem when dealing with native code which can use the whole 64Gb address space.
UPD
According to the JVM Anatomy Quark #23: Compressed References the applications will use Non-Zero Based Mode which allows to use up to 32Gb and doing the 3 bit shift. That's ok because the pointers still will be 32-bit but in fact if the MaxRAM for the app is 4Gb then we can skip the 3-bit shift.
But this looks like not needed optimization because anyway it will use the same instruction in form 0xc(%r12,%r11,8),%eax but instead of 8 it will be 0.
For experiments we can run java -version itself as any other java app with VM options to set it's memory limit.
For example we can set -XX:MaxRAM=2147483648 or -Xmx2G i.e. limit max memory to 2 Gibibyte.
To see the real COOPS mode we can add the option -Xlog:gc+heap+coops=info:
$ java -XX:MaxRAM=2147483648 -XX:MaxRAMPercentage=100 -Xlog:gc+heap+coops=info -version
[0.011s][info][gc,heap,coops] Heap address: 0x0000000080000000, size: 2048 MB, Compressed Oops mode: 32-bit
openjdk version "12.0.2" 2019-07-16
OpenJDK Runtime Environment Zulu12.3+11-CA (build 12.0.2+3)
OpenJDK 64-Bit Server VM Zulu12.3+11-CA (build 12.0.2+3, mixed mode, sharing)
Here we can see that COOPS are enabled and it use 32-bit mode, i.e. all pointers are 4 byte long. In the same time note Heap address: 0x0000000080000000 where the hex address 0000000080000000 corresponds to 2147483648 i.e. 2Gb. This value is called like "heap base" or HeapBaseMinAddress final flag or NarrowOopHeapBaseMin in sources and is 2Gb by default but may be changed via flag.
That means that while 32-bit in enough to address 4Gb but the first 2Gb will be reserved by the HeapBaseMinAddress so only 2Gb left for 32-bit mode.
If you try to set -XX:MaxRAM=2147483649 i.e. higher on the one byte than 2Gb (2147483648) then the 32-bit mode will be disabled and it will be used "Zero based, Oop shift amount: 3" mode instead:
$ java -XX:MaxRAM=2147483649 -XX:MaxRAMPercentage=100 -Xlog:gc+heap+coops=info -version
[0.010s][info][gc,heap,coops] Heap address: 0x000000077fe00000, size: 2050 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
What is unclear for me here is that the Heap address 0x000000077fe00000 is 32210157568 i.e. 29,998046875 Gb while on my current computer where I do the test I have only 16Gb or RAM.
In the same time if we specify HeapBaseMinAddress flag then it will be used as the Heap address:
$ java -XX:MaxRAM=2147483648 -XX:MaxRAMPercentage=100 -Xlog:gc+heap+coops=info -XX:HeapBaseMinAddress=8g -version
[0.008s][info][gc,heap,coops] Heap address: 0x0000000200000000, size: 2048 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
Here 0x0000000200000000 corresponds to 8589934592 i.e. 8Gb.
Also I wrote a simple app that runs as a daemon and runned two instances of it and the both started in 32-bit mode and the both share the same Heap address:
$ java -XX:MaxRAM=2147483648 -XX:MaxRAMPercentage=100 -XX:+UnlockDiagnosticVMOptions -Xlog:gc+heap+coops=info DaemonThreadTest
[0.011s][info][gc,heap,coops] Heap address: 0x0000000080000000, size: 2048 MB, Compressed Oops mode: 32-bit
I don't know how this is possible because I thought that COOPS 32-bit mode is possible only if the app can take 3rd and 4th gibibytes of the whole address space but if first daemon already took it then the second should use some other available space out of the first 4Gb. I'm confused here.
Anyway, I'll continue my test and learning on next wee on the computer with the 64Gb of RAM. Thank you for your replies.
All available modes can be seen from the universe.hpp
// For UseCompressedOops
// Narrow Oop encoding mode:
// 0 - Use 32-bits oops without encoding when
// NarrowOopHeapBaseMin + heap_size < 4Gb
// 1 - Use zero based compressed oops with encoding when
// NarrowOopHeapBaseMin + heap_size < 32Gb
// 2 - Use compressed oops with disjoint heap base if
// base is 32G-aligned and base > 0. This allows certain
// optimizations in encoding/decoding.
// Disjoint: Bits used in base are disjoint from bits used
// for oops ==> oop = (cOop << 3) | base. One can disjoint
// the bits of an oop into base and compressed oop.
// 3 - Use compressed oops with heap base + encoding.
enum NARROW_OOP_MODE {
UnscaledNarrowOop = 0,
ZeroBasedNarrowOop = 1,
DisjointBaseNarrowOop = 2,
HeapBasedNarrowOop = 3,
AnyNarrowOopMode = 4
};
