2

for context, im running a modded minecraft server, and one of the mods (computercraft) creates a new thread for every block you place. The server only uses around 15gb of ram during runtime, but it has over 30gb available for use (-Xms8192M -Xmx32000M), so the OOME makes no sense really. I did read up somewhere that an OOME is thrown if there's too many live threads and you try to create another one, which is what makes me think there's a limit

whenever the live thread count (which is visible using VisualVM) hits 500 or more, it throws an OutOfMemoryException when trying to start a new thread. I couldn't find much on the internet (i found one post, but that mentioned 503 threads but nothing they mentioned seemed to help, and my server can't even get past 500)

I though this might be an issue with Docker maybe (im using pterodactyl) so i set docker's "TasksMax" variable to Infinity, but that didn't seem to help. i've tried lowering the stack size (-Xss320k before the Xms bits) which didn't help

I've also checked the system memory usage in case there are any memory leaks, and there aren't. its only using around 50% ram. An when i run the ulimit -a command in linux, this is what it shows:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 256816
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256816
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

An example when a new thread is created, is when a new player joins (as each player has their own NetConnection). This is what the crash stack trace looks like:

java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method)
    at java.lang.Thread.start(Thread.java:717)
    at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1378)
    at org.bukkit.craftbukkit.v1_6_R3.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:360)
    at net.minecraft.server.MinecraftServer.func_71190_q(MinecraftServer.java:831)
    at net.minecraft.server.dedicated.DedicatedServer.func_71190_q(DedicatedServer.java:332)
    at net.minecraft.server.MinecraftServer.func_71217_p(MinecraftServer.java:782)
    at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:665)
    at net.minecraft.server.ThreadMinecraftServer.run(SourceFile:583)

Does anyone know what could be causing this thread limit of 500? the only way i've fixed it is by force killing (thread.interrupt() then thread.stop()) the newest created computercraft threads (they aren't that important to the server's ability to run, but it inconveniences players having to reboot the computers. and im not exactly sure what kind of monitors/locks computercraft uses in those threads so it could be dangerous by killing them)

I've also tried the explanation at "java.lang.OutOfMemoryError : unable to create new native Thread" but that didnt seem to help either

edit: just incase it helps someone else, i didn't find a fix (to increase the thread limit). I think it might be a limit with what i use to run the server (something to do with docker). I found out it was a problem with computercraft's parallel API (aka multithreading) so i had to disable that (by editing the pastebin api, and the text editor save/exit feature)

REghZY
  • 69
  • 1
  • 7
  • 3
    A separate thread for each block is kind of insane, you almost never want more concurrent threads than you have cores available. Even if it didn't crash out, the context switching between thousands of threads would grind your server to a cpu-melting halt =) – Mike 'Pomax' Kamermans Aug 10 '21 at 17:40
  • Or try using virtual threads in Project Loom, where having so many threads *is* reasonable (as long as the threads are not doing work that is CPU-bound). But Loom is still experimental, not available in official releases of Java/JDKs. – Basil Bourque Aug 10 '21 at 17:47
  • i didn't make the mod :-) but yea a separate thread for each block is a bit much tbh. But even with around 300 of them in loaded chunks, it still managed to stay around 20tps – REghZY Aug 10 '21 at 17:48
  • @BasilBourque - https://www.loom.com/blog/cost-of-context-switching :-) :-) – Stephen C Aug 23 '21 at 07:52

0 Answers0