3

I am novice to java memory issues and facing problems while debugging the memory usage of a java process and need your help. As per "top", the java process is taking 8G of Resident memory and 9.4G of virtual memory. This is too much for what the process does and I want to check why it is eating away the memory. So I took a heap dump using jmap (without the 'live' option as it may trigger GC) and found that only 100MB of live objects are present and there are 1.5G of unreachable objects. So where is the rest of 6G memory going? What are the other types of non-heap memory i should check? Also, can you help me with any tools / unix commands for find this out?

JVM args : -Xms is 2G and -Xmx is 8G

Please let me know if I should add more information here to give you more context.

Thanks in Advance.

shadowfax
  • 971
  • 1
  • 10
  • 17
  • Sounds like a memory leak, I do not know what can cause this behavior in Java, but maybe i'll find something. Can you post some code or elaborate on what the purpose of this program is and what it does? – UnTraDe Dec 18 '14 at 14:38
  • It receives data from a service, transforms it from a custom format to json and writes to another web service. It happens in real time and doesn't keep the data in memory apart from the short duration it takes to transform. It keeps opening and closing connections with both the source service and target services. – shadowfax Dec 18 '14 at 14:49
  • See http://stackoverflow.com/questions/26041117/growing-resident-memory-usage-rss-of-java-process – Lari Hotari Sep 07 '16 at 09:18

2 Answers2

1

You told Java it could take 8Gb and it did. Perhaps you increased the memory usage inside the JVM to 8Gb for a short while; then it garbage collected and threw away 6.4Gb worth of unreachable objects. But the JVM doesn't return that memory to the system.

However, when you run out of physical memory on your machine, these 6.4Gb of unused pages will be swapped out. You just haven't run out of memory yet in your machine so they're still part of the RSS. At least, that's my educated guess.

See also: Is there a way to lower Java heap when not in use?

Community
  • 1
  • 1
Erwin Bolwidt
  • 30,799
  • 15
  • 56
  • 79
  • Thanks for your response. You mean to say, even if GC has thrown away 6.4G of data, resident memory will still show 8G since i allocated so much? Is there a way to find out the exact memory used then, live + unreachable, excluding the unused memory? – shadowfax Dec 18 '14 at 15:12
  • Well if my thinking is right, then it should be removed from your resident set as soon as there is another process that needs physical memory. But if you don't have other processes that need the memory, it would stay there. – Erwin Bolwidt Dec 18 '14 at 15:33
  • Ok. Any suggestions on how i can find out if this is really the case (unused pages) or if there is non-heap memory that is being used? – shadowfax Dec 18 '14 at 15:38
  • No nifty tools that I know of. But you could start your java process that takes 8Gb again (or write something else that just allocates memory, whether in Java or a small C program that does a big malloc and then sleeps for a bit) and see if that reduces the RSS of your first process. – Erwin Bolwidt Dec 18 '14 at 15:40
  • this is incorrect, java will return the memory even if xmx is 8g it will not return xms which is 2g... and its easy to test just run a simple program and give it 8g xmx then watch for a minute on visual vm – vach Dec 25 '18 at 12:30
1

On Linux this might give some useful information:

pmap -x PID

see man pmap for more about pmap

You should also be aware that there is a known problem on Linux with glibc >= 2.10 (Ubuntu >= 10.04, RHEL >= 6) that causes "lost memory".

The cure is to set export MALLOC_ARENA_MAX=4

Search for MALLOC_ARENA_MAX on Google or StackOverflow for more information about that.

You might want to tune also other malloc options to optimize for low fragmentation of allocated memory:

# tune glibc memory allocation, optimize for low fragmentation
# limit the number of arenas
export MALLOC_ARENA_MAX=2
# disable dynamic mmap threshold, see M_MMAP_THRESHOLD in "man mallopt"
export MALLOC_MMAP_THRESHOLD_=131072
export MALLOC_TRIM_THRESHOLD_=131072
export MALLOC_TOP_PAD_=131072
export MALLOC_MMAP_MAX_=65536
Lari Hotari
  • 5,190
  • 1
  • 36
  • 43