How do goroutines work? (or: goroutines and OS threads relation)

Question

How can other goroutines keep executing whilst invoking a syscall? (when using GOMAXPROCS=1)
As far as I'm aware of, when invoking a syscall the thread gives up control until the syscall returns. How can Go achieve this concurrency without creating a system thread per blocking-on-syscall goroutine?

From the documentation:

Goroutines

They're called goroutines because the existing terms—threads, coroutines, processes, and so on—convey inaccurate connotations. A goroutine has a simple model: it is a function executing concurrently with other goroutines in the same address space. It is lightweight, costing little more than the allocation of stack space. And the stacks start small, so they are cheap, and grow by allocating (and freeing) heap storage as required.

Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management.

Answer 1

If a goroutine is blocking, the runtime will start a new OS thread to handle the other goroutines until the blocking one stops blocking.

Reference : https://groups.google.com/forum/#!topic/golang-nuts/2IdA34yR8gQ

Answer 2

Ok, so here's what I've learned: When you're doing raw syscalls, Go indeed creates a thread per blocking goroutine. For example, consider the following code:

package main

import (
        "fmt"
        "syscall"
)

func block(c chan bool) {
        fmt.Println("block() enter")
        buf := make([]byte, 1024)
        _, _ = syscall.Read(0, buf) // block on doing an unbuffered read on STDIN
        fmt.Println("block() exit")
        c <- true // main() we're done
}

func main() {
        c := make(chan bool)
        for i := 0; i < 1000; i++ {
                go block(c)
        }
        for i := 0; i < 1000; i++ {
                _ = <-c
        }
}

When running it, Ubuntu 12.04 reported 1004 threads for that process.

On the other hand, when utilizing Go's HTTP server and opening 1000 sockets to it, only 4 operating system threads were created:

package main

import (
        "fmt"
        "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
        http.HandleFunc("/", handler)
        http.ListenAndServe(":8080", nil)
}

So, it's a mix between an IOLoop and a thread per blocking system call.

Answer 3

It can't. There's only 1 goroutine that can be running at a time when GOMAXPROCS=1, whether that one goroutine is doing a system call or something else.

However, most blocking system calls, such as socket I/O, waiting for a timer are not blocked on a system call when performed from Go. They're multiplexed by the Go runtime onto epoll, kqueue or similar facilities the OS provides for multiplexing I/O.

For other kinds of blocking system calls that cannot be multiplexed with something like epoll, Go does spawn a new OS thread, regardless of the GOMAXPROCS setting (albeit that was the state in Go 1.1, I'm not sure if the situation is changed)

Answer 4

You have to differentiate processor number and thread number: you can have more threads than physical processors, so a multi-thread process can still execute on a single core processor.

As the documentation you quoted explain, a goroutine isn't a thread: it's merely a function executed in a thread that is dedicated a chunk of stack space. If your process have more than one thread, this function can be executed by either thread. So a goroutine that is blocking for a reason or another (syscall, I/O, synchronization) can be let in its thread while other routines can be executed by another.

Answer 5

I’ve written an article explaining how they work with examples and diagrams. Please feel free to take a look at it here: https://osmh.dev/posts/goroutines-under-the-hood

Answer 6

From documentation of runtime:

The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit.

so when one OS thread is blocked for syscall, another thread can be started - and GOMAXPROCS = 1 is still satisfied. The two threads are NOT running simultaneously.

Answer 7

Hope this example of sum numbers helps you.

package main

import "fmt"

func put(number chan<- int, count int) {
    i := 0
    for ; i <= (5 * count); i++ {
        number <- i
    }
    number <- -1
}

func subs(number chan<- int) {
    i := 10
    for ; i <= 19; i++ {
        number <- i
    }
}

func main() {
    channel1 := make(chan int)
    channel2 := make(chan int)
    done := 0
    sum := 0

    //go subs(channel2)
    go put(channel1, 1)
    go put(channel1, 2)
    go put(channel1, 3)
    go put(channel1, 4)
    go put(channel1, 5)

    for done != 5 {
        select {
        case elem := <-channel1:
            if elem < 0 {
                done++
            } else {
                sum += elem
                fmt.Println(sum)
            }
        case sub := <-channel2:
            sum -= sub
            fmt.Printf("atimta : %d\n", sub)
            fmt.Println(sum)
        }
    }
    close(channel1)
    close(channel2)
}