1

I created lots of goroutines on MacOs, and there was an error emitted when program executing.

goRoutineId = 3710, i = 3683, len(chan) = 2049
runtime: failed to create new OS thread (have 2049 already; errno=12)
fatal error: runtime.newosproc

So I wonder what the "failed to create new OS thread" means, is that an operating system limitation, of just golang has no ability to create more goroutine? Thank you for helping me.

leo
  • 1,045
  • 3
  • 15
  • 27
  • This may happen if you have a lot of goroutines that are all blocked at the same time, ie. in system calls. Those threads do not count towards the GOMAXPROCS limitations, therefore more will be spawned. What are your goroutines doing? – SirDarius Sep 29 '17 at 08:54
  • My goroutine just simply return a number. And my temporary solution is to limit the number of goroutines executing concurrently. When my program was executing, I found that the incrementation memory is no more than 200M, and I still have very enough memory which is not used yet, so I still wonder whether golang itself has memory limitation – leo Sep 29 '17 at 09:05
  • 1
    If you don't know why you have more than 2049 goroutines, get a full stack trace and look at what's there. You have blocked goroutines somewhere that are causing you to spawn more threads. – JimB Sep 29 '17 at 12:32

1 Answers1

3

It's OS's limitation. I would assume you are using linux.

According to the source of go, it is calling clone system call

ret := clone(cloneFlags, stk, unsafe.Pointer(mp), unsafe.Pointer(mp.g0), unsafe.Pointer(funcPC(mstart)))
sigprocmask(_SIG_SETMASK, &oset, nil)

if ret < 0 {
    print("runtime: failed to create new OS thread (have ", mcount(), " already; errno=", -ret, ")\n")
    if ret == -_EAGAIN {
        println("runtime: may need to increase max user processes (ulimit -u)")
    }
    throw("newosproc")
}

from the manpage of clone(2), when the errno=12, the error reason is out of memory

ENOMEM Cannot allocate sufficient memory to allocate a task structure
              for the child, or to copy those parts of the caller's context
              that need to be copied.
ymonad
  • 11,710
  • 1
  • 38
  • 49
  • I keep monitoring memory usage percent when program executing, and I found that the incrementation memory is no more than 200M, and I still have very enough memory which is not used yet, so I still wonder whether golang itself has memory limitation. – leo Sep 29 '17 at 08:36
  • 1
    @PengWu Even when there's enough memory, there might be a case for limitation using `ulimit` or some other reason. read: https://stackoverflow.com/questions/1367373/python-subprocess-popen-oserror-errno-12-cannot-allocate-memory/13329386#13329386 – ymonad Sep 29 '17 at 09:02
  • Didn't find the mapping for errno and it's explanation in `man clone`. They are in `/usr/include/asm-generic/errno-base.h` – realcp1018 Apr 25 '23 at 03:44