Why does this small Java program make MacOS restart?

Question

Code is as follows

Set<Thread> threads = new HashSet<>();

Runnable r = () -> {
    try {
        Thread.sleep(Long.MAX_VALUE);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
};

for (int i = 0; i < 20000; i++) {
    Thread t = new Thread(r);
    threads.add(t);
    t.start();
    if (i % 100 == 0) {
        System.out.println(i);
    }
    Thread.sleep(2);
}

When executed, I start seeing values like

as expected, and it goes until I see:

3900
4000
Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:717)
    at App.main(scratch.java:24)
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated

But then after a short while (10 - 20 seconds or so) MacOS decides to restart. What is the cause for the restart I am seeing here? The main thread throwing an exception, but the process having ~4000 threads sleeping causes ... what in the operating system? Is this a memory overflow or related to task scheduler of the OS?

MacOS version: 10.14.3 (18D109)
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

If the app crashed it wouldn't be that big a problem. If a program is able to trigger a system restart however, it is a severe security relevant operating system bug(but could also be a hardware e.g. memory problem - don't blame the os for every malfunction). — kai, Feb 16 '19 at 15:11
Have you checked the journal/event log? (I suppose that OS offers some for of it). Might help to go through it...? — ernest_k, Feb 16 '19 at 15:11
@Carcigenicate This is whyI specifically notes I am on MacOS. I think it is more related to the underlying OS this program runs on and not the JVM itself. I think it can only be reproduced in MacOS. (I can not try Windows as I do not have a Windows machine at hand.) — Koray Tugay, Feb 16 '19 at 15:18
Currently trying on Windows for kicks. IntelliJ takes like a year to start. — Carcigenicate, Feb 16 '19 at 15:19
a stackoverflow in contrast to a bufferoverflow can. because a stackoverflow means writting to physical memory outside the virtual memory of the process - that is any memory: eg a drivers memory. — kai, Feb 16 '19 at 15:22
@ernest_k I am looking at something called `system.log` but can not find anything. Most likely I am not looking at the right place. — Koray Tugay, Feb 16 '19 at 15:22
Can confirm it crashes on MacOS Mojave, java version, "1.8.0_202-ea", doesn't seem to crash on Windows 10 (I used a different machine though). — Sash Sinha, Feb 16 '19 at 15:26
@Carcigenicate `This doesn't even OOME ` Wow! It lets you create 20000 sleeping Threads? What happens when you try with a larger number? — Koray Tugay, Feb 16 '19 at 15:28
@KorayTugay I removed the upper bound. I'm at 40k right now. — Carcigenicate, Feb 16 '19 at 15:29
It got stuck at 65552 threads for like a minute, then continued on. I wonder if running this in IntelliJ is effecting anything? (90k by the time I finished writing this comment). Anyways I'll quit commenting as this doesn't appear to effect me (now it's stuck at 90k) — Carcigenicate, Feb 16 '19 at 15:34
@Carcigenicate I do not know honestly.. Looks like Windows lets you spawn as many threads as you want in a process whereas MacOS limits it to 4096.. (For me at least). — Koray Tugay, Feb 16 '19 at 15:35
LOL. At 100k threads I got a Blue Screen of Death, and it restarted. Maybe that counts as a repro? — Carcigenicate, Feb 16 '19 at 15:45
@KorayTugay proof. Here is an example for an stackoverflow exploit from 2016, that breaks a linux kernel(RIP) : https://www.exploit-db.com/exploits/39992 — kai, Feb 16 '19 at 15:52
Two interesting notes: (1) I can reproduce the KP without the OOME when creating fewer threads (the panic will occur on my Mac when threads > 3K), and (2) attaching VisualVM and monitoring the heap indicates the JVM is no where near running out of memory. — sherb, Feb 23 '19 at 23:18

Badaro · Answer 1 · 2019-02-25T17:08:53.843

11

Despite the fact that the console shows the program has finished, the JVM process still runs until all resources are released. Meanwhile, your OS is out of threads, slow and unstable, which cause lagging in all processes, including the JVM finalization. As a self defense, the OS triggers a kernel panic. And that is why your MacOS restarts.

*OS - Operating System

edited Feb 25 '19 at 17:08

answered Feb 19 '19 at 21:26

Badaro

576
3
12

Soutzikevich · Answer 2 · 2019-02-25T21:06:43.670

Java was built in the 90's, when there were only multi-core processors.

Surely Java has evolved, as have modern-day processors. Nowadays we have 8-core processors, with large caches (e.g: 12MB).

Even though concurrent processing has evolved much, Java is still designed around the 1-core processor model. But, enough with history, let me explain very very simply what happens.

Just by simply creating a new thread in Java, we waste a lot of memory.

Every thread consumes around ~ 512KB - 1MB, depending on your JVM version (see how much memory a thread takes in java and Java Thread: Retained Memory). Bearing this in mind, when continuously creating new Threads, at some point they will consume all of the heap's memory.

Now, I have never tried this on my own, but I assume that your computer's operating system shuts down/restarts due to the "out of memory" error, as a countermeasure. (This is much like the triple fault, that caused the infamous "Blue Screen of Death" on Windows, where the machine needed to restart to reset the CPU's state)

One possible solution for this, is to manually set the maximum heap size to be used by the JVM. Thus, when your program completely utilises the pre-allocated heap, it will not cause a shutdown. Please refer to this SO question on how to do this.

score 1 · Answer 3 · answered Jul 07 '20 at 14:53

I just stumbled on the same problem today. Here's some python code triggering the same kernel panic on 10.15.5 (Catalina). Tested it on two macs to make sure this is not a hardware problem:

https://github.com/ephes/django_async/blob/master/measure_threads_memory.py

Maybe I go and write a bug report.

score 1 · Answer 4 · answered Oct 20 '21 at 19:45

Cause of the restart is a counter measure by OS, when OS becomes CPU heavy and threads consumes the OS resources. To be specific,"unable to create new native thread" specifies that OS is out of threads and can't create further threads.

Also, to be noted, A JVM runs in a single process and threads in a JVM share the heap belonging to that process. Java will utilize the underlying OS threads to do the actual job of executing the code on different CPUs, if running on a multi-CPU machine. When each Java thread is started, it creates an associated OS thread and the OS is responsible for scheduling, etc.

OS is smart enough to make use of available cores for thread execution based on priority and other scheduling algorithms.

And, "java.lang.OutOfMemoryError" signifies, that OS has utilized the heap memory allocated or as specified in JVM heap size.

So, if the heap size is large and available memory is not enough for other processes in OS causing the lag and eventually in order to reset the CPU state, OS restarts.

ErikWi · Answer 5 · 2019-02-24T05:42:19.660

0

This is a Fork bomb variant. This can cause severe slowdown, but no user program should be able to crash the OS. This is probably a bug in the OS or memory error. Try running a memory check?

edited Feb 24 '19 at 05:42

answered Feb 23 '19 at 20:34

ErikWi

514
5
11

score -4 · Answer 6 · answered Feb 16 '19 at 17:16

-4

Most likely because you either haven't given your JVM enough memory, or your computer hardware and macOS combination do not permit that many threads to be active at once. This problem is not limited to macOS but some Linux distributions, e.g. Bodhi Linux, have this limitation too. Do not be deceived by "OutOfMemoryError" - this can often mean the JVM was unable to allocate a native thread.

answered Feb 16 '19 at 17:16

Sina Madani

1,246
3
15
27

6

I do not think `do not permit that many threads to be active at once` is not a reason for a restart. – Koray Tugay Feb 16 '19 at 17:37
3

If it would a "do not permit", it where not such a problem. In the ops case, that doesn't happen, the "do not permit" of the os is somehow bypassed and a situation arises in which the os itself breaks. This is a severe security problem: an unpriviledged user(at least in terms of process isolation) is able to trigger a priviledged operation. – kai Feb 16 '19 at 21:03
Fair point. I wonder what happens if running the program with a lower priority and also setting the thread priorities in the program to the lowest. – Sina Madani Feb 17 '19 at 07:06

Why does this small Java program make MacOS restart?

6 Answers6