1

We recently got an OutOfMemory is due to "unable to create new native thread" on our Tomcat servers. We know what the issue is - there was a bug that spawned too many Threads beyond what the ulimit on the box allowed.

What intrigues me is that, when this happened the JVM didn't dump the heap. I checked that is behavior was consistent in Java 6 and 8.

I wrote some quick Groovy PoCs to prove this behavior. So while I have used groovy here the behavior is consistent with my Servlet application (just easier for me to share a working model of the problem).

Dumps the Heap - TraditionalOOM.groovy:

byte[] b = new byte[1000000000]

Does NOT dump the heap - ManyThreads.groovy:

int MAX = 5000;

for (int i = 0; i < MAX; i++) {

    new Thread() {
        public void run() {
            sleep(1000);
        }
    }.start();
}

Thread.activeCount();

The Java Options relevant to this issue (IMHO) are the following 2:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/somewhere

I wanted to know:

  • Why does the JVM not dump the heap during an OOM (even dumping the thread dump would be helpful)?
  • Is the answer the obvious one, that JVM doesn't have a thread to perform the dump?

Update

I got my answer - seems like this is the expected behavior and Oracle (or Sun) had rejected this behavior as a defect - http://bugs.java.com/bugdatabase/view_bug.do;jsessionid=74ee28dae329645479a51f6c78f2?bug_id=6784422

Community
  • 1
  • 1
Y123
  • 915
  • 13
  • 30

1 Answers1

0

I got my answer - seems like this is the expected behavior and Oracle (or Sun) had rejected this behavior as a defect -

http://bugs.java.com/bugdatabase/view_bug.do;jsessionid=74ee28dae329645479a51f6c78f2?bug_id=6784422

The link states "Allocation of a native thread is not from the heap or the permanent generation - hence failure to allocate one does not result in a heap dump. (There would be little point in dumping the heap as that is not what was exhausted.)"

Which I don't completely agree with, since the heap dump also has a wealth of info on the threads. The JVM should at least dump the thread dump if that is Oracle's (or then Sun's) position.

Y123
  • 915
  • 13
  • 30
  • 1
    I have never done it with the Oracle/Sun JVM, but I know that it's possible to configure the IBM JVM to drop either javacore or heap dumps on fairly arbitrary conditions like particular exceptions – Alex Fitzpatrick Dec 08 '15 at 02:31
  • That's interesting, mind sharing the option you use with the IBM VM? I may be able to look for something on the lines for Oracle VM. Thanks. – Y123 Dec 08 '15 at 03:58