6

I got a new VPS to run some java programs that me and some buddies have made. I start the process with a line like this:

java -Xmx512M -jar program.jar

On our old VPS, you could use the 'top' command to see how much virtual and resident memory was being used. It would use like 600-700mb of virtual memory. Now on our new VPS, with that same command, the virtual memory seems to always be an extra ~2gb over the -Xmx value. So instead of the virtual memory being around 600-700mb, it's instead 2700-3000mb.

The old VPS is running CentOS 5.7 and the new is running CentOS 6.2. Both are running JRE 1.7u3 64bit.

Why is this and how can I fix it?

edit: top

PID   USER    PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
27645 pyro    20   0 3003m 270m  10m S  5.0  1.7   1:19.18 java -Xmx512M -jar cserver.jar

another edit: I am not questioning why virtual memory is using more memory than specified in the java command line. I am questioning why it is using so much more than it used to.

Reed D.
  • 61
  • 4
  • 1
    Check that your java process definitely is using the Xmx512m parameter by using 'ps -ef', and then if it really is, update your question with the output of top. – Rich Mar 12 '12 at 07:41
  • If the resident size is 270MB and there is no paging issue, is there a problem? – Sean Owen Mar 12 '12 at 08:26
  • If the program hasn't changed any since the old VPS, and with the old VPS, the Virtual memory would only use ~600-700mb, then yes something doesn't seem right. Both the old and the new VPS run the same OS and the same Java version. – Reed D. Mar 12 '12 at 08:28
  • No change in OS. No change in JVM version. Did the applicaiton code change? – Samarth Bhargava Mar 12 '12 at 09:13
  • A very similar thread, has got a great answer. http://stackoverflow.com/questions/561245/virtual-memory-usage-from-java-under-linux-too-much-memory-used – Samarth Bhargava Mar 12 '12 at 09:27
  • I was informed that there actually was a difference in the OS's (CentOS 5.7 and CentOS 6.2). I believe to have found the cause as an updated glibc version that RHEL6 uses. Thank you very much for the link, learned a lot about virtual memory from that answer. – Reed D. Mar 12 '12 at 10:08

2 Answers2

4

The heap is not the only thing which consumes virtual memory. Virtual memory is the amount of address space the application has rather than the amount of memory it is using (the resident is a better indicator)

Virtual memory includes all the thread stack space, direct memory and memory mapped files.

The first thing I would check is the number of threads your application uses, the more threads, the more virtual memory.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • I am aware of the virtual memory thing. I am just curious as to why it is allocating so much memory on this new VPS. Both of the VPS's are running the same version of Java (1.7u3) and the same OS (CentOS 6.2). As well as this, the program hasn't changed any since the VPS switch. – Reed D. Mar 12 '12 at 08:31
  • If you run `pmap -x {pid}`. Often when people say nothing has changed, it means they don't know of anything which has changed, but clearly something has. ;) – Peter Lawrey Mar 12 '12 at 08:33
  • Thanks for the pmap command. Apparently I was misinformed, one of my buddies told me that our old VPS was actually CentOS 5.7 and not 6.2. Found out that the 'issue' (which isn't an issue at all) is with glibc 2.11's anonymous heap mapping. – Reed D. Mar 12 '12 at 09:18
  • So this could be the cause. Its is still working out if this is a problem or just mis-leading. pmap should help you there. Its possible the default thread stack size is much larger resulting in much more virtual memory being reserved (which may not be a problem either) – Peter Lawrey Mar 12 '12 at 09:20
  • I think it is the issue, comparing the old vps' pmap to the new for the same exact process, the new shows more [anon] mappings, of which each taking up ~64mb of virtual. The few [anon] in the old vps' pmap only took up ~1mb. I'm going to assume it's from the updated glibc version because of the 64mb blocks. The pmap command helped me identify this, thank you!! – Reed D. Mar 12 '12 at 10:01
  • I would try specifying the thread stack size like `-XX:ThreadStackSize=512` to see if these regions change size. – Peter Lawrey Mar 12 '12 at 12:10
1

Virtual memory usage means how much address space is used and does not necessarily translate directly into RAM usage. Stack, mapped files (including binaries and libraries), etc. all contribute to virtual memory but not always to the RAM actually used. Note that your RES (resident in RAM) memory usage is a rather nice 270 MB only. On a 32-bit machine, you may run into address space limitations, so there virtual memory is a scarce resource if you get close to the 2 GB mark (the value may also be 1 GB or 3 GB depending on OS). On a 64-bit system, virtual memory (address space) is close to unlimited, so by itself, a high value need not be treated as a risk. Of course, if it is also related to actual high RAM usage or lots of mapped files which you cannot find out why are used, it's worth looking into.

Of course, JVM also has some actual overhead (in physically allocated memory), related to garbage collector's house keeping, compiler's work, native code etc., and this will also be reflected in virtual memory usage. But since the RES and SHR positions are not very high, I'd say there's no reason to panic, especially if you're 64-bit.

Michał Kosmulski
  • 9,855
  • 1
  • 32
  • 51