0

All,

I have a Java Memory Process which starts to take more and more memory with time. To put a Cap on the heap space usage I set -Xmx option to 512M. Slowly over a period of time the Process memory usage reached 2GB.

I have analyzed the code for possible memory leaks using various tools like MAT and YOURKIT and found no such possible leaks in the Java code. The code also makes use of One Native function which looks leak free.

I have the following questions :

  1. Is it possible to cause an upper limit on the total memory a Java Process can use ?
  2. What are the other memory usages of the JVM apart from the Heap ?
  3. Does Linux work on 'Working set' memory model of Windows where a process when put into background releases it's memory.
  4. Even after using -Xmx option with 512M shouldn't the JVM throw an 'Out Of memory' if the heap usage increases. This makes me suspect that the memory is being leaked by something else apart from the Heap space. From the memory dumps it seems that the Heap Memory is not increasing.

thanks for the answer.

Geek
  • 23,089
  • 20
  • 71
  • 85
  • 1
    Strong suggestion: Become familiar with the [JConsole](http://help.eclipse.org/indigo/index.jsp?topic=%2Forg.eclipse.mat.ui.help%2Fgettingstarted%2Fbasictutorial.html) tool (or equivalent for your JVM) and see if it can shed light on what's happening: http://docs.oracle.com/javase/1.5.0/docs/guide/management/jconsole.html – paulsm4 Sep 24 '12 at 07:36
  • 1
    Are you using a lot of operations on strings ? When you modify a String , a new object is created (not true with StringBuffer). The old one is not immediately destroyed. The jvm keep him around for a while, because you may need it again. – Perello Sep 24 '12 at 07:44
  • Current Java versions generate StringBuilders automatically for String concatenation for you behind the scene. Also prefer StringBuilders (unsynchronized) to StringBuffers whenever possible! – user643011 Sep 24 '12 at 07:51
  • Perello : Yes, I am using a lot of Strings and there are a bunch of not collected strings in the heap. The question is if I have an upper limit on the heap size, why does the Process take more and more memory. Why does it not do OutOfMemory and stop. I strongly feel that something other than the heap is taking memory. – Geek Sep 24 '12 at 08:32

3 Answers3

5

How are you measuring memory usage?

Usually there are 3 kinds of memory usage:

  • VIRT -- Virtual Memory Size (KiB) The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out and pages that have been mapped but not used.
  • RES -- Resident Memory Size (KiB) The non-swapped physical memory a task has used.
  • SHR -- Shared Memory Size (KiB) It simply reflects memory that could be potentially shared with other processes.

The virtual memory size can become larger (several GB) than what you specified with -Xmx, but this does not do any harm. RES plus SHR is what you should be looking at.

Apart from the heap there is another class of memory usage not affected by -Xmx (permgen). But that is usually capped at a few MB. You might want to read the HotSpot GC Tuning Guide.

user643011
  • 2,163
  • 3
  • 21
  • 38
  • I am talking about the RES memory here. – Geek Sep 24 '12 at 08:33
  • 1
    Also see http://stackoverflow.com/questions/12292368/jvm-exceeds-maximum-memory-defined-with-xmx?rq=1: - lots and lots of thread stacks, - memory-mapped files that are not being closed when they should be, - some native code library using (possibly leaking) out-of-heap memory. – user643011 Sep 24 '12 at 08:42
1

Let me augment my "comment" with a full-fledged "response".

If you're doing a lot of string manipulation, you should definitely not be using the "String" class. And whatever's going on, Java has many great tools (including, but not limited to, JConsole) that let you intelligently analyze the problem. Your Two New BFFs should be:

  • StringBuilder (manipulate strings efficiently) and

  • JConsole (study your program's behavior; including heap allocation and garbage collection).

And, as user643011 correctly pointed out, there are lots of other things that might conceivably "leak": including files not being closed, lots of threads using lots of stack space, etc. etc.

Here is one other good link:

Community
  • 1
  • 1
paulsm4
  • 114,292
  • 17
  • 138
  • 190
  • Hi Paul, I agree with your reasoning. My problem however is that I am not getting an Out Of Memory Error and my Heap space is merely 512 MB which is much less than the actual process size. So there is something else which takes up 1.5 G. – Geek Sep 25 '12 at 07:17
  • 1) I hear you that the problem isn't "heap" per se, 2) that's why I cited these particular links: they should give you plenty of tools and plenty of tips, 3) look especially for "not closing/not freeing", "#/threads" and "recursion", and *CAREFULLY CHECK THE METHOD THROWING THE EXCEPTION* (and carefully check its stack traceback!): http://stackoverflow.com/questions/4512147/how-to-debug-java-outofmemory-exceptions – paulsm4 Sep 25 '12 at 16:18
1

If you suspect memory leak in your java application, you should definitely try Plumbr. It is quite good and spotting them and gives quite useful reports.

Nikem
  • 5,716
  • 3
  • 32
  • 59