Go - Execute a Bash Command n Times using goroutines and Store & Print its result

Question

I'm pretty new to Golang in general and I'm trying to execute a bash command with its arguments n times, then, store the Output in a variable and Print it.

I'm able to do it just one time, or just using loops like the following:

package main

import (
    "fmt"
    "os/exec"
    "os"
    "sync"
)


func main() {

    //Default Output
    var (
        cmdOut []byte
        err    error
    )

    //Bash Command
    cmd  := "./myCmd"
   //Arguments to get passed to the command
   args := []string{"arg1", "arg2", "arg3"}

    //Execute the Command
    if cmdOut, err = exec.Command(cmd, args...).Output(); err != nil {
        fmt.Fprintln(os.Stderr, "There was an error running "+cmd+" "+args[0]+args[1]+args[2], err)
        os.Exit(1)
    }
    //Store it
    sha := string(cmdOut)
    //Print it
    fmt.Println(sha)
}

This works just fine, I'm able to read the output easily.

Now, I would like to repeat this very same operation for n times, using goroutines.

I tried following the very same approach of the guy who answered How would you define a pool of goroutines to be executed at once in Golang? but I'm not able to make it work.

That's what I tried so far:

package main

import (
    "fmt"
    "os/exec"
    "sync"
)


func main() {

    //Bash Command
    cmd  := "./myCmd"
    //Arguments to get passed to the command
     args := []string{"arg1", "arg2", "arg3"}

    //Common Channel for the goroutines
    tasks := make(chan *exec.Cmd, 64)

    //Spawning 4 goroutines
    var wg sync.WaitGroup
    for i := 0; i < 4; i++ {
        wg.Add(1)
        go func() {
            for cmd := range tasks {
                cmd.Run()
            }
            wg.Done()
        }()
    }

    //Generate Tasks
    for i := 0; i < 10; i++ {
        tasks <- exec.Command(cmd, args...)
        //Here I should somehow print the result of the latter command
    }
    close(tasks)

    // wait for the workers to finish
    wg.Wait()

    fmt.Println("Done")

}

But, I don't really find out how to store the i-result of an executed command and print it.

How can I achieve this?

Thanks in advance, for any clarification on the question just leave a comment.

Answer 1

so the following fixes your problem

• you can call Cmd.Output() to get the output of the command. otherwise you could connect the Cmd.StdOutPipe pipe to a byte.Buffer for example and read from that.

• your goroutine logic was wrong. it would only execute 4 times cause then the waitgroup would be Done() and main would exit. You are correct to close the channel from main to signal the workers to exit the for range loop. wg.Done should be called after that so i defered it.

• when you execute anonymous functions with the go command try not to capture anything from the parent scope. it can lead to disaster. instead figure out which parameters you want and pass them to the function.

`

package main 

import (
    "fmt"
    "os/exec"
    "sync"
)
func main() {
    cmd := "./foo.sh"
    //Arguments to get passed to the command
    args := []string{"bar", "baz"}

    //Common Channel for the goroutines
    tasks := make(chan *exec.Cmd, 64)

    //Spawning 4 goroutines
    var wg sync.WaitGroup
    for i := 0; i < 4; i++ {
            wg.Add(1)
            go func(num int, w *sync.WaitGroup) {
                    defer w.Done()
                    var (
                            out []byte
                            err error
                    )
                    for cmd := range tasks { // this will exit the loop when the channel closes
                            out, err = cmd.Output()
                            if err != nil {
                                    fmt.Printf("can't get stdout:", err)
                            }
                            fmt.Printf("goroutine %d command output:%s", num, string(out))
                    }
            }(i, &wg)
    }
    //Generate Tasks
    for i := 0; i < 10; i++ {
            tasks <- exec.Command(cmd, args...)
    }
    close(tasks)

    // wait for the workers to finish
    wg.Wait()

    fmt.Println("Done")

}

`