2

I am running a Java application on a Linux-Cluster with SLURM as resource manager. To run my application I have to specify for SLURM the amount of memory I will need. SLURM will run my application in a kind of VM with the specified amount of memory. To tell my java application how much memory it can use I use the "-Xmx##g" parameter. I choose it 1GB less than I have requested from SLURM.

My problem is that I am exceeding the amount of memory I have chosen on SLURM and it terminates my application. It seems that the JVM uses about 1GB of memory, probably for things like GC or so.

Is there a possibility to restrict the size of the JVM or at least to tame it.

Cheers, Markus

markus
  • 57
  • 1
  • 8
  • Are you using many threads? They need memory for their stacks (1mb is default I think) and that might not be limited by the Xmx parameter. – Janick Bernet Sep 22 '13 at 10:27
  • possible duplicate of [How to set the maximum memory usage for JVM?](http://stackoverflow.com/questions/1493913/how-to-set-the-maximum-memory-usage-for-jvm) – Thomas Sep 22 '13 at 10:29
  • @inflagranti I am using 6 Threads. By an default size of 1mb probably not enough to explain why it is crashing. – markus Sep 22 '13 at 10:35
  • @Thomas I have seen this question, but the answer there is: Use the "-Xmx##" parameter, which I am already use. – markus Sep 22 '13 at 10:37

1 Answers1

3

The maximum heap setting only limited the maximum heap. There are other memory regions which you have not limited such as

  • thread stacks
  • perm gen
  • shared libraries
  • native memory used by libraries
  • direct memory
  • memory mapped files.

If you want to limit the over all memory usage you need to be clear about whether you are limiting virtual memory or resident memory. Often monitoring tools make the mistake of monitoring virtual memory which shows a surprising lack of understanding of how applications work, or even why you monitor an application in the first place.

You want to monitor resident memory usage which means you need to know how much memory your application uses over time apart from the heap, then work out how much heap you can have plus some margin for error.

. To tell my java application how much memory it can use I use the "-Xmx##g" parameter. I choose it 1GB less than I have requested from SLURM.

At a guess I would start with 1/2 GB with -Xmx512m and see what is the peak resident memory and increase it if you find there is always a few hundred MB head room.

BTW 1 GB of memory doesn't cost that much these days (as little as $5). Your time could be worth much more than the resources you are trying to save.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130