How to limit the number of goroutines in recursion？

Question

I'm using goroutine to traverse all files in the specified directory. I want to know how to limit the number of goroutines under recursive traversal

I wrote this code, but it prompted panic chan send `

package main

import (
    "fmt"
    "os"
    "runtime"
    "sync"
    "time"
)

var (
    wg    sync.WaitGroup
    // limit = make(chan bool, 20)
    // totalchan = make(chan bool)
)

func Work(path string, size int, finished bool) {
    fmt.Printf("gn1111: %d\n", runtime.NumGoroutine())

    defer func() { wg.Done() }()

    fl, err := os.ReadDir(path)
    if err == nil {
        for _, file := range fl {
            if file.IsDir() {
                // limit <- true
                wg.Add(1)
                go Work(path+file.Name()+"/", size, false)
            } else {
                fmt.Printf("gn2222: %d\n", runtime.NumGoroutine())
            }
        }
    }

    // if !finished {
    //  <-limit
    // }
}

func main() {
    
    path := "C:/Windows/"
    size := 0
    start := time.Now()
    wg.Add(1)
    go Work(path, size, true)
    wg.Wait()
    fmt.Printf("total= %d, cost: %v\n", 0, time.Since(start))
}

`

Use a [worker pool](https://stackoverflow.com/questions/38170852/is-this-an-idiomatic-worker-thread-pool-in-go/38172204#38172204) and just submit the tasks using a channel. — icza, Nov 28 '22 at 09:07
[Here](https://stackoverflow.com/questions/71766816/loop-through-all-files-in-all-folders-recursively-as-fast-as-possible-in-golang) you can find another example for this purpose. — Fenistil, Nov 28 '22 at 09:45

score 0 · Answer 1 · answered Nov 28 '22 at 10:26

the errgroup has a Limiting functionality

SetLimit limits the number of active goroutines in this group to at most n. A negative value indicates no limit. Any subsequent call to the Go method will block until it can add an active goroutine without exceeding the configured limit. The limit must not be modified while any goroutines in the group are active.

How to limit the number of goroutines in recursion？

1 Answers1