Golang, processes and shared memory

Question

Today a friend of mine told me that Go programs can scale themselves on multiple CPU cores. I were quite surprised to hear that knowing that system task schedulers do not know anything about goroutines and hence can't run them on multiple cores.

I did some search and found out that Go programs can spawn multiple OS tasks to run them on different cores (the number is controlled by GOMAXPROCS environment variable). But as far as I know forking a process leads to complete copy of process data and different processes run in different address spaces.

So what about global variables in Go programs? Are they safe to use with multiple goroutines? Do they somehow synchronize between system processes? And if they do then how? I am mainly concerned about linux and freebsd implementations.

[The official documentation of goroutines](https://golang.org/doc/effective_go.html#goroutines) should be helpful. — chrk, Dec 25 '16 at 15:36
As well as [this article](https://www.goinggo.net/2014/01/concurrency-goroutines-and-gomaxprocs.html). — chrk, Dec 25 '16 at 15:43
@chrk, the problem is: not the official documentation nor the second article shed light on how go runtime provides synchronization of shared variables. The official documentation states: "A goroutine has a simple model: it is a function executing concurrently with other goroutines in the same address space". But they can't be in the same address space if they are in different processes... — ea7ababe, Dec 25 '16 at 16:03
Where are you getting the idea that Go uses separate processes for anything? — JimB, Dec 25 '16 at 16:09
Let's read it again more carefully then.It states:"They're called goroutines because the existing terms—threads, coroutines, **processes**, and so on-convey inaccurate connotations"; so don't assume they're processes. As far as the address space is concerned, it states that "Goroutines are multiplexed onto multiple **OS threads**".I don't aim to provide a complete answer here, but what you really need to study in order to figure out the answers to your questions by yourself, is how threads work in Linux; then test:`strace`,`/proc//*` and see some answers here in SO for more advanced stuff — chrk, Dec 25 '16 at 16:21
@JimB, well, [this answer](http://stackoverflow.com/a/25390444/4769561) says that linux task scheduler can only run a process on one CPU core at a time. This means that a go program must use multiple OS tasks to utilize multiple cores. And that means different address spaces for each task. Am I wrong? — ea7ababe, Dec 25 '16 at 16:21
A thread, in linux, is also known as a Light Weight Process (http://www.tldp.org/FAQ/Threads-FAQ/Types.html). Each thread can be on a separate core and has it's own stack, but shares the address space of the spawning process (shared memory) — David Budworth, Dec 25 '16 at 16:33
@DavidBudworth, o-o-oh, so Linux does have support for threads with shared memory... I never heard about that. Well that clears up everything. — ea7ababe, Dec 25 '16 at 16:43

ea7ababe · Accepted Answer · 2016-12-25T23:35:59.980

I figured it out! It's all in go sources.

There is a Linux system call that I were unaware of. It's called "clone". It is more flexible than fork and it allows a child process to live in its parent's address space.

Here is a short overview of the thread creation process.

First there is a newm function in src/runtime/proc.go. This function is responsible for creating a new working thread (or machine as it is called in comments).

// Create a new m. It will start off with a call to fn, or else the scheduler.
// fn needs to be static and not a heap allocated closure.
// May run with m.p==nil, so write barriers are not allowed.
//go:nowritebarrier
func newm(fn func(), _p_ *p) {

    // ... some code skipped ...

    newosproc(mp, unsafe.Pointer(mp.g0.stack.hi))
}

This function calls newosproc which is OS-specific. For Linux it can be found in src/runtime/os_linux.go. Here are relevant parts of that file:

var (
    // ...

    cloneFlags = _CLONE_VM | /* share memory */
        _CLONE_FS | /* share cwd, etc */
        _CLONE_FILES | /* share fd table */
        _CLONE_SIGHAND | /* share sig handler table */
        _CLONE_THREAD /* revisit - okay for now */
)

// May run with m.p==nil, so write barriers are not allowed.
//go:nowritebarrier
func newosproc(mp *m, stk unsafe.Pointer) {

    // ... some code skipped ...

    ret := clone(cloneFlags, /* ... other flags ... */)

    // ... code skipped
}

And the clone function is defined in architecture-specific files. For amd64 it is in src/runtime/sys_linux_amd64.s. It is the actual system call.

So Go programs do run in multiple OS threads which enables spanning across CPUs, but they use one shared address space.

Phew... I love Go.

Golang, processes and shared memory

1 Answers1