Collect errors from goroutines nested in loops

Question

I'm trying to collect errors from goroutines in loop, but dont't understand how it must correctly work https://go.dev/play/p/WrxE0vH6JSG

func init() {
    rand.Seed(1500929006430687579)
}

func goroutine(n int, wg *sync.WaitGroup, ch chan error) {
    defer wg.Done()
    defer fmt.Println("defer done")

    fmt.Println("num ", n)
    if n == 1 {
        ch <- fmt.Errorf("error")
    } else {
        ch <- nil
    }
}

func main() {
    var wg sync.WaitGroup
    var err error
    errs := make(chan error)
    platforms := 2
    types := 3
    for j := 0; j < platforms; j++ {
        wg.Add(1)
        for k := 0; k < types; k++ {
            wg.Add(1)
            n := rand.Intn(2)
            go goroutine(n, &wg, errs)
        }

        for k := 0; k < types; k++ {
            wg.Add(1)
            n := rand.Intn(2)
            go goroutine(n, &wg, errs)
        }
    }
    wg.Wait()
    err = <-errs
    fmt.Println(err)
}

how should I collect array of errors correctly and done all wait groups?

A solution like [this](https://go.dev/play/p/-345msbtSZN) will let you collect the errors concurrently while also ensuring they're all collected. — Gavin, Nov 11 '22 at 20:51
```go run -race .```: ```================== Found 1 data race(s) exit status 66``` in line ```go goroutine(n, errs)``` — SaneQ, Nov 14 '22 at 13:21

score 1 · Accepted Answer · answered Nov 11 '22 at 21:14

In Golang channels is similar to pipes in bash (|). But in contrast to bash pipes which are used to transport output of one command to input of another command, Go channels are used to transport some data between goroutines. You can read more about channels here. Channels have capacity. When you don't specify capacity for the channel go assumes that it has 0 capacity. Channels with zero capacity often called unbuffered channels while channels with non-zero capacity called buffered. When channel is full (number of elements in channel is equal to channel's capacity) than all write operations on the channel (->errs) block execution flow until read operation (<-errs) will be presented.

In your particular example you have unbuffered channel (the channel with 0 capacity). Thus any write operation (->errs) on your channel will block the execution until some read operation would be provided, therefore all goroutines that you launched will be blocked despite the only one goroutine that would be able to proceed write operation when the flow of the main function moved forward to read operation (err = <-errs).

To solve your issue you could create one extra goroutine that would read from channel concurrently with goroutines that would write to channel. It will look like that:

func init() {
    rand.Seed(1500929006430687579)
}

func goroutine(n int, wg *sync.WaitGroup, ch chan error) {
    defer fmt.Println("defer done")
    defer wg.Done()

    fmt.Println("num ", n)
    if n == 1 {
        ch <- fmt.Errorf("error")
    }
}

func main() {
    var wg sync.WaitGroup
    errs := make(chan error)
    platforms := 2
    types := 3

    go func() {
        for e := range errs {
            fmt.Println(e)
        }
    }()

    for j := 0; j < platforms; j++ {
        for k := 0; k < types; k++ {
            wg.Add(1)
            n := rand.Intn(2)
            go goroutine(n, &wg, errs)
        }

        for k := 0; k < types; k++ {
            wg.Add(1)
            n := rand.Intn(2)
            go goroutine(n, &wg, errs)
        }
    }
    wg.Wait()
}

In addition you have several bugs and inaccuracies that I refactored in your code:

You shouldn't write nil in channel with errors. If you want errs chan to comprise only errors so write there only if your function executed with non-nil error.
You had one extra wd.Add(1) as the beginning of j loop so there was disbalance between Add functions and Done function. 3.
Furthemore, you add defer fmt.Println("defer done") after defer wg.Done() but defers constructions are executed in reversed order than they were specified so it would be more correct to put defer fmt.Println("defer done") before defer wg.Done() so that "defer done" would really signalize that all previous defers had been executed.

How can i save all errors in a slice? Append inside goroutine will cause race condition, I think — SaneQ, Nov 14 '22 at 13:15
Race conditions occurs when you have multiple goroutines and one of them is writing some data (its not matter which operation perform other gororuines, its enough condition to data race when just one goroutine is writiing). If you will write your erros inside goroutine instead of `fmt.Println(e)` you could wait its finishing with `WaitGroup` otherwise if you want to read this slice in your main function concurrently with goroutine that writes to this slice than you should use mutex. — cooleck, Nov 14 '22 at 14:43

Collect errors from goroutines nested in loops

1 Answers1