2

I used following code to test the max number of thread, but it's very queer.

import java.util.concurrent.atomic.AtomicInteger;

public class TestThread extends Thread {
    private static final AtomicInteger count = new AtomicInteger();

    public static void main(String[] args) {
        try{
            while (true) {
                (new TestThread()).start();
            }
        } catch (Error | Exception e) {
            System.out.println(e.getMessage());
        }
    }

    @Override
    public void run() {
        System.out.println(count.incrementAndGet());
        while (true)
            try {
                Thread.sleep(Integer.MAX_VALUE);
            } catch (InterruptedException e) {
                break;
            }
    }
}

OS is ubuntu:18.04, the memory is 4G, number of processors is 2. JDK: openjdk1.8.0_252

java -Xss1m -Xms1024m -Xmx1023m TestThread

java -Xss512k -Xms1024m -Xmx1023m TestThread

java -Xss1m -Xms2048m -Xmx2048m TestThread

In the result, the number of threads are always equal around 10000th.

  1. why the result is the same.
  2. I set the -Xss1m in the command, does it mean each thread will have alone 1m memory? if it has alone memory, 10000th threads will have 10G memory, actually there just 4G memeory in the machine.
Freedom
  • 61
  • 7
  • 1. What error message does it log? 2. How much total memory do you have (ram+swap)? – Joni Jul 13 '20 at 14:12
  • The error message is OOM unable to create new native thread. I print the top in ubuntu. KiB Mem:4015860 total, 1923636 free, 1429424 used, 662800 buff/cache; KiB Swap: 1942896 total, 1942896 free, 0 used, 2336928 avail Mem – Freedom Jul 13 '20 at 14:27
  • ulimit -u is 15511 – Freedom Jul 13 '20 at 14:30
  • 1
    [Maximum number of threads per process in Linux?](https://stackoverflow.com/questions/344203/maximum-number-of-threads-per-process-in-linux) – akuzminykh Jul 13 '20 at 15:10
  • This is interesting, there are several things that can restrict the number of threads you can create, but it's hard to say which one you are hitting. Do you see a hs_err_pid log file generated? – Joni Jul 13 '20 at 15:29
  • I can't see the hs_err_pid log, the program is noranl except the OOM – Freedom Jul 14 '20 at 13:08

1 Answers1

5

-Xss determines the maximum stack size for Java threads. When a thread starts, the operating system creates a region of virtual memory for the stack of this size, but the memory is not immediately allocated in physical RAM.

The physical pages are allocated lazily on the first memory access to the page. As long as your threads do not call deep methods, they do not use all the stack space. That's why only a few bottom stack pages are allocated in RAM, no matter what -Xss is set.

Linux enables memory overcommit by default. This means, it allows to commit more virtual memory than the amount of physical RAM (as long as this virtual memory is not actually backed by physical pages, like in your case).

So, in your case OutOfMemoryError is thrown not because of stack space, but due to one of the other OS limits:

  • ulimit -u
  • sysctl kernel.pid_max
  • sysctl kernel.threads-max

See also:

apangin
  • 92,924
  • 10
  • 193
  • 247
  • 1
    Excellent answer! The last link you mentioned (https://stackoverflow.com/questions/344203/maximum-number-of-threads-per-process-in-linux) also talks about `/proc/sys/vm/max_map_count` - this is documented in the kernel docs (https://www.kernel.org/doc/Documentation/sysctl/vm.txt) as "maximum number of memory map areas a process may have" but I'm not sure how this corresponds to the max number of threads a process can create - do you have any details about that? Is there a minimum number of "maps" that is created for every thread that a process creates? – Juraj Martinka Jul 14 '20 at 10:53
  • 1
    @JurajMartinka Usually there are two memory mappings for each thread stack: one for accessible stack area, and another for [guard pages](https://pangin.pro/posts/stack-overflow-handling). By chance, the accessible stack area can be combined with an adjacent mapping that has the same attributes (anonymous mapping with RW access), but generally `max_map_count` should be strictly more than 2x expected number of threads. – apangin Jul 14 '20 at 12:22
  • thanks @apangin, need more time to view these message. – Freedom Jul 14 '20 at 13:13