Java program running out of RAM but not really

Question

I'm running my application from within IntelliJ for this experiment and it's crashing with this error:

java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:717)
    at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
    at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1025)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

or

java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:717)
    at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
    at okhttp3.Dispatcher.enqueue(Dispatcher.java:130)
    at okhttp3.RealCall.enqueue(RealCall.java:100)
    at retrofit2.OkHttpCall.enqueue(OkHttpCall.java:107)
    at com.pubnub.api.endpoints.Endpoint.async(Endpoint.java:127)
    at com.pubnub.api.managers.SubscriptionManager.startSubscribeLoop(SubscriptionManager.java:275)
    at com.pubnub.api.managers.SubscriptionManager.access$400(SubscriptionManager.java:31)
    at com.pubnub.api.managers.SubscriptionManager$5.onResponse(SubscriptionManager.java:324)
    at com.pubnub.api.managers.SubscriptionManager$5.onResponse(SubscriptionManager.java:275)
    at com.pubnub.api.endpoints.Endpoint$1.onResponse(Endpoint.java:201)
    at retrofit2.OkHttpCall$1.callSuccess(OkHttpCall.java:138)
    at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:117)
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153)
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

I fired up Java Visual VM and this is what I see:

The error happened around the vertical line. I'm puzzled by this because the used memory didn't get that close to the available memory and the available memory is only about half the max that's allowed. Also, during this exercise, my computer had more than 10GB of actual RAM free. This is running on a Windows 10 computer.

I created this little program to explore reaching the limit of threads:

public class ThreadBlower {
    public static void main(String[] args) {
        System.out.println("Creating threads until we crash.");
        for (int i = 0; ; i++) {
            Thread thread = new Thread(() -> { while (true) { } });
            thread.setName(String.format("Thread %d", i));
            thread.start();
            System.out.printf("Running %d threads.%n", i);
        }
    }
}

and even though it did crash with java.lang.OutOfMemoryError: unable to create new native thread it took about 2700 threads to get there. I don't think my app gets anywhere near that number.

Any ideas what's going on here?

This may or may not be related: this seems to happen only with Java 32 bit and not 64 bit. What also is happen only with 32 bit is this: EXCEPTION_ACCESS_VIOLATION on "JavaFX Application Thread"

Suggested duplicate: [“java.lang.OutOfMemoryError : unable to create new native Thread”](https://stackoverflow.com/questions/16789288/java-lang-outofmemoryerror-unable-to-create-new-native-thread) — Erwin Bolwidt, Apr 19 '18 at 08:43
@ErwinBolwidt: I think that question is likely very related, but both my development and my targeted platform is Windows, so, there might or might not be different techniques and/or solutions to the problem. — Pablo Fernandez, Apr 19 '18 at 08:50
How do you know okhttp3 didn't create 2700 threads? Have you made a thread dump? — rustyx, Apr 19 '18 at 09:41
@rustyx: I don't really, but Java Visual VM is saying that Live Peak was 67 (its in the middle of the screenshot). — Pablo Fernandez, Apr 19 '18 at 09:42
@rustyx: would I learn anything from a thread dump that I'm not learning from Java Visual VM chart and list of threads? — Pablo Fernandez, Apr 19 '18 at 09:43
@Berger how would I know that? I have an HTTP client using Spring's RestTemplate, but I'm also loading WebView. — Pablo Fernandez, Apr 19 '18 at 09:45
Read this: https://square.github.io/okhttp/3.x/okhttp/okhttp3/OkHttpClient.html — rustyx, Apr 19 '18 at 09:53
@rustyx: I'm not using okhttp directly, I'm using Spring's RestTemplate of which I'm creating one instance and re-using it. Every now and then they do get re-created. When I'm causing this crash in my app I created two `RestTemplate` objects. — Pablo Fernandez, Apr 19 '18 at 10:01
RestTemplate uses commons-httpclient by default. So you must be configuring okhttp3 and pubnub somewhere. And why do you recreate it? — rustyx, Apr 19 '18 at 10:53
I'm not doing anything directly with okhttp3, but yes, I do use pubnub and javafx's webview. I create a `RestTemplate` when the user logs in. So, if they log out and log in again, it gets re-created. — Pablo Fernandez, Apr 19 '18 at 11:00
without the code that spawns the thread of workers, I cannot determine the exact cause of the problem. It could be that your application is requesting more memory than the default JVM reserved memory space. You can expand it with an additional arg command though. — KarelG, Apr 20 '18 at 13:27
@KarelG: there are many thread pools in this app, only one controlled by me directly. I'm already assigning 1GB to the app and it's crashing way before using it up. You can see the values on the top left of the screenshot. — Pablo Fernandez, Apr 20 '18 at 13:28
@pupeno I am talking about memory per thread, aka "thread stacks" in java. I think that the problem is there, but I am not sure about it because I do not know -1- how you are spawning your threads and -2- what each thread does (eg is there any concurrent activity ?) — KarelG, Apr 20 '18 at 14:03

score 0 · Answer 1 · answered Apr 19 '18 at 08:44

0

java.lang.OutOfMemoryError: unable to create new native thread can be caused by exhaustion of file descriptors or OS native threads.

If you are on Linux you can run

$ ulimit -a

to check what are the current limits for your user as per man ulimit.

answered Apr 19 '18 at 08:44

Karol Dowbecki

43,645
9
78
111

Thank you for the answer @Karol. I'm running on Windows 10, so, no `ulimit` for me. I added some details about how many threads I have to the question. I'm not sure what the limits are for Windows. – Pablo Fernandez Apr 19 '18 at 08:49
@Timir: although I appreciate Karol's answer, his answer focuses on Linux while the problem I'm having is on Windows. I added more information to the question about how many threads I'm creating and it's not that many. – Pablo Fernandez Apr 19 '18 at 09:11
Don't raise `ulimit` to avoid the error. The thread limit is already huge. Fix the underlying issue instead. – rustyx Apr 19 '18 at 09:43
@rustyx: well, first, I'm using Windows and so do my customers, so, no `ulimit`; second, I can't go to each person that downloads my app and change the setting of their computer. I would agree though about not raising `ulimit` on a Linux server. – Pablo Fernandez Apr 19 '18 at 09:49
@pupeno although I'm not familiar with Windows [this answer](https://stackoverflow.com/a/481919/1602555) suggests that the total thread limit will depend on stack space. It's my understanding that all Windows processes will compete for this stack space, thus it might depend on what else is running on this machine. – Karol Dowbecki Apr 19 '18 at 17:41

Java program running out of RAM but not really

1 Answers1