1

I have a java application that has a weird memory usage, I noticed that the memory consumption is significantly higher than the max heap size (Xmx is 800m, usage is 1.4g).

One of the recent changes preceding this was a large increase in the unique strings in use, so I thought maybe the many strings I have use a lot of memory outside the heap - is this possible?

I'm running java 11.

EDIT:

for example, in this article it mentions:

When we create a String object using the new() operator, it always creates a new object in heap memory. On the other hand, if we create an object using String literal syntax e.g. “Baeldung”, it may return an existing object from the String pool, if it already exists.

Giving the impression of 2 different areas - the heap and the string pool.

orirab
  • 2,915
  • 1
  • 24
  • 48

3 Answers3

4

In JDK, the string pool that contains interned strings, consists of two parts:

  1. The hash table, which is allocated off-heap and contains the references to the interned strings. This memory area is shown in Native Memory Tracking report as "String table".
  2. The interned strings contents. These are regular java.lang.String objects in Java heap.

So, the string pool has both on-heap and off-heap parts. The off-heap part is typically smaller, but it still can take a significant amount of extra memory, if the applications creates too many interned strings.

Use Native Memory Tracking to find the amount of memory consumed by the String table.

See this answer to find other reasons why the process may take much more memory than -Xmx.

apangin
  • 92,924
  • 10
  • 193
  • 247
  • I'll take a look at this tomorrow, but it looks awesome! Ty very much! – orirab Apr 05 '20 at 21:34
  • I second Andrei's answer to this question which is really useful: https://stackoverflow.com/questions/53451103/java-using-much-more-memory-than-heap-size-or-size-correctly-docker-memory-limi/53624438#53624438 – Juraj Martinka Apr 06 '20 at 08:06
  • accepted, this is an awesome answer. explains the main question, suggests how to verify, and offers a suggestion to the underlying problem – orirab Apr 06 '20 at 09:40
2

Since Java 7, the string pool has been a part of the regular heap (see here). So the regular heap size also constrains the size of the string pool.

The real thing you should be taking from that article is that new String(...) always creates a new String object that is distinct from all previously created String objects. That is not the case for a String object that corresponds to a string literal.

I noticed that the memory consumption is significantly higher than the max heap size (Xmx is 800m, usage is 1.4g).

The explanation for that is that there are various ways that a JVM uses memory that is not part of the regular heap. These include:

  • the memory for the JVM executable and its shared libraries
  • memory used for native code libraries loaded by the application
  • memory segments used for Java thread stacks
  • memory used to represent the metaspace ... which holds JIT compiled native code and so on,
  • memory in the "native" heap allocated by native code; e.g. using malloc.
  • memory allocated for direct buffers
orirab
  • 2,915
  • 1
  • 24
  • 48
Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
  • TY for the answer! can you link to any official documentation? – orirab Apr 05 '20 at 13:37
  • 1
    https://blogs.oracle.com/poonam/about-g1-garbage-collector,-permanent-generation-and-metaspace – Stephen C Apr 05 '20 at 13:41
  • And my memory was faulty. The string pool moved to the regular heap in Java 7 not Java 8. – Stephen C Apr 05 '20 at 13:44
  • Ty for the link, I'll look at it in a bit. Regarding the additions to your answer, I'm aware of the additional parts of the JVM and outside memory, but I have a hard time believing 600m or more are in use outside the heap for these klind of things. – orirab Apr 05 '20 at 13:45
  • Believe it. (Maybe you are unaware of what your application and its dependent libraries are doing behind the scenes.) – Stephen C Apr 05 '20 at 13:46
-1

You can use https://visualvm.github.io/index.html to analyse and inspect for example your occupied memory and heap.

smithnblack
  • 505
  • 7
  • 17