When will Go scheduler create a new M and P?

Question

Just learned golang GMP model, now I understand how goroutines, OS threads, and golang contexts/processors cooperate with each other. But I still don't understand when will an M and P be created?

For example, I have a test code to run some operations on DB and there are two test cases (two batches of goroutines):

func Test_GMP(t *testing.T) {
    for _ = range []struct {
        name string
    }{
        {"first batch"},
        {"second batch"},
    } {
        goroutineSize := 50
        done := make(chan error, goroutineSize)
        for i := 0; i < goroutineSize; i++ {
            go func() {
                // do some databases operations...
                // each goroutine should be blocked here for some time...

                // propogate the result
                done <- nil
            }()
        }

        for i := 0; i < goroutineSize; i++ {
            select {
            case err := <-done:
                assert.NoError(t, err)
            case <-time.After(10 * time.Second):
                t.Fatal("timeout waiting for txFunc goroutine")
            }
        }
        close(done)
    }
}

In my understanding, if M is created in need. In the first batch of goroutines, 8 (the number of virtual cores on my computer) OS threads will be created and the second batch will just reuse the 8 OS threads without creating new ones. Is that correct?

Appreciate if you can provide more materials or blogs on this topic.

score 3 · Accepted Answer · answered Jul 09 '21 at 07:09

M is reusable only if your processes are not blocking or not any sys-calls. In your case you have blocking tasks inside your go func(). So, number of M will not be limited to 8 (the number of virtual cores on my computer). First batch will block and remove from P and wait for blocking processes get finished while new M create an associate with P.

We create a goroutine through Go func ();

There are two queues that store G, one is the local queue of local scheduler P, one is the global G queue. The newly created G will be saved in the local queue in the P, and if the local queues of P are full, they will be saved in the global queue;

G can only run in m, one m must hold a P, M and P are 1: 1 relationship. M will pop up a executable G from the local queue of P. If the local queue is empty, you will think that other MP combinations steals an executable G to execute;

A process executed by M Scheduling G is a loop mechanism;

When M executes syscall or the remaining blocking operation, M will block, if there are some g in execution, Runtime will remove this thread M from P, then create one The new operating system thread (if there is an idle thread available to multiplex idle threads) to serve this P;

When the M system call ends, this G will try to get an idle P execute and put it into this P's local queue. If you get P, then this thread m becomes a sleep state, add it to the idle thread, and then this G will be placed in the global queue.

1. P Quantity:

The environment variable $ GomaxProcs is determined by the Runtime method gomaxprocs () when the environment variable is scheduled. After GO1.5, GomaxProcs will be set by default to the available cores, and before default it is 1.This means that only $ GOMAXPROCS Goroutine is run at the same time at any time executed.

2. M quantity:

The GO language itself limits: When the GO program starts, the maximum number of M will set the maximum number of M. However, the kernel is difficult to support so many threads, so this limit can be ignored. SetMaxThreads function in runtime / debug, set the maximum number of M A M blocking, you will create new M.

The number of M and P has no absolute relationship, one m block, p will create or switch another M, so even if the default number of P is 1, there may be many M out.

Please refer following for more details,

When will Go scheduler create a new M and P?

1 Answers1

Linked