How to check that java heap is fragmented?

Question

I have a 29G heap dump, created by Hotspot VM after OutOfMemoryError occurred. Heap analyzer (I use YourKit) showed that all objects (including unreachable) take 26G. My guess is remaining 3G are wasted because the heap is fragmented. Is there a way to check this theory?

Does heap fragmentation really happen with a modern, generational, garbage collector? — Raedwald, Jul 05 '19 at 14:50
See https://stackoverflow.com/questions/17051126/why-do-i-get-outofmemory-when-20-of-the-heap-is-still-free — Raedwald, Jul 05 '19 at 14:56
@Raedwald thanks for links. Heap is collected from JRE 8; unfortunately I don't have VM parameters in dump, so GC is either Parallel GC or G1. — Sergey Fedorov, Jul 05 '19 at 15:06

score 3 · Answer 1 · answered Jul 05 '19 at 15:42

This has nothing to do with heap fragmentation.

The difference in size is explained by the heap dump format. Note that the heap dump is not the raw contents of the heap - it is a serialized representation of the contents in HPROF format.

So, each object instance is represented by a HPROF_GC_INSTANCE_DUMP record of the following structure:

u1   record tag
u8   object ID                             
u4   stack trace serial number             
u8   class ID                       
u4   number of bytes that follow           
u1*  <field data>

Consider an instance of java.lang.Integer. In 64-bit Compressed OOPs HotSpot JVM an instance of Integer takes 16 bytes: 8 bytes header + 4 bytes class pointer + 4 bytes value field. The same instance will occupy 29 bytes in a heap dump, thus producing huge 81.25% overhead.

Of course, the heap in a typical Java application is usually filled with much bigger objects, i.e. arrays. That's why the average overhead of HPROF record headers appears smaller, somewhere about 10%, just like in your case.

But the heap analyzing tools do not show the object's occupation in the hprof file, but basically *guess* the missing information, like pointer sizes, alignment/padding, etc., when calculating the object size. So the problem is indeed the hprof format, but not the way, objects are represented, but the information about the actual heap organization that is missing. For completeness: https://shipilev.net/blog/2014/heapdump-is-a-lie/ — Holger, Jul 08 '19 at 11:08

How to check that java heap is fragmented?

1 Answers1