What is the cause of the deadlock in my Go code using WaitGroups and Buffered Channels?

Question

WaitGroups, Buffered Channels, and Deadlocks

I have this bit of code which results in a deadlock and I'm not certain why. I have tried using mutex locking in a few different places, closing channels in and outside of separate go routines, but the result is still the same.

I'm trying to send data through one channel (inputChan), and then read it from another (outputChan)

package main

import (
    "fmt"
    "sync"
)

func listStuff(wg *sync.WaitGroup, workerID int, inputChan chan int, outputChan chan int) {
    defer wg.Done()

    for i := range inputChan {
        fmt.Println("sending ", i)
        outputChan <- i
    }
}

func List(workers int) ([]int, error) {
    _output := make([]int, 0)

    inputChan := make(chan int, 1000)
    outputChan := make(chan int, 1000)

    var wg sync.WaitGroup
    wg.Add(workers)

    fmt.Printf("+++ Spinning up %v workers\n", workers)
    for i := 0; i < workers; i++ {
        go listStuff(&wg, i, inputChan, outputChan)
    }

    for i := 0; i < 3000; i++ {
        inputChan <- i
    }

    done := make(chan struct{})
    go func() {
        close(done)
        close(inputChan)
        close(outputChan)
        wg.Wait()
    }()

    for o := range outputChan {
        fmt.Println("reading from channel...")
        _output = append(_output, o)
    }

    <-done
    fmt.Printf("+++ output len: %v\n", len(_output))
    return _output, nil
}

func main() {
    List(5)
}

LeGEC · Accepted Answer · 2023-06-02T06:52:43.397

The code in your main function is sequential and first tries to write 3k values into inputChan then will read values from outputChan.

Your code blocks on the first of those steps:

nothing drains from outputChan before 3k values are succesfully sent to inputChan, so the workers end up stuck on outputChan <- i after the first 1k value
once the workers stop draining from inputChan, main will get stuck on inputChan <- i after ~2k values

One way to fix this can be to have the producer (inputChan <- i) and the end consumer (for o := range outputChan {) run in separate goroutines.

You can keep one of these actors in the main goroutine, and spin a new one for the other. For example :

go func(inputChan chan<- int){
    for i := 0; i < 3000; i++ {
        inputChan <- i
    }
    close(inputChan)
}(inputChan)

done := make(chan struct{})
go func() {
    close(done)
    // close(inputChan) // I chose to close inputChan above, don't close it twice
    close(outputChan)
    wg.Wait()
}()

...

https://go.dev/play/p/doBgfkAbyaO

one extra note: the order of actions around signaling done is important ; channels done and outputChan should only be closed after wg.Done() indicates that all workers are finished

    // it is best to close inputChan next to the code that controls
    // when its input is complete.
    close(inputChan)
    // If you had several producers writing to the same channel, you
    // would probably have to add a separate waitgroup to handle closing,
    // much like you did for your workers

    go func() {
        wg.Wait()
        // the two following actions must happen *after* workers have
        // completed
        close(done)
        close(outputChan)
    }()

Sumail Xu · Answer 2 · 2023-06-08T07:28:48.670

package main

import (
    "fmt"
    "sync"
)

func listStuff(wg *sync.WaitGroup, workerID int, inputChan chan int, outputChan chan int) {
    defer wg.Done()

    for i := range inputChan {
        fmt.Println("sending ", i)
        outputChan <- i
    }
}

func List(workers int) ([]int, error) {
    _output := make([]int, 0)

    inputChan := make(chan int, 1000)
    outputChan := make(chan int, 1000)

    var wg sync.WaitGroup
    wg.Add(workers)

    fmt.Printf("+++ Spinning up %v workers\n", workers)
    for i := 0; i < workers; i++ {
        go listStuff(&wg, i, inputChan, outputChan)
    }

    done := make(chan struct{})

    //read as the same time as send
    go func() {
        for o := range outputChan {
            fmt.Println("reading from channel...")
            _output = append(_output, o)
        }
        close(done)
    }()

    for i := 0; i < 3000; i++ {
        inputChan <- i
    }

    fmt.Println("**************")

    go func() {

        close(inputChan)
        wg.Wait()
       //close after send finished
        close(outputChan)
    }()

    <-done
    fmt.Printf("+++ output len: %v\n", len(_output))
    return _output, nil
}

func main() {
    List(5)
}

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). — Community, Jun 02 '23 at 07:15

What is the cause of the deadlock in my Go code using WaitGroups and Buffered Channels?

2 Answers2