Capturing main process stdout into bash variable, keep printing subprocess output to console?

Question

I'm developing a CLI. The CLI writes to stdout that I'd like to capture in a bash variable. That is, I'd like to be able to do this:

output=$(my_cli_command)

My CLI also runs a subprocess in that command. I'd like the output of the subprocess to be printed to the terminal when output=$(my_cli_command) is executed.

What do I have to do when running the subprocess to ensure that it (1) is printed to the console but (2) is not captured into the output variable? Or is this not possible?

I'm writing my CLI in Go. I've put a reproducible example here:

package main

import (
    "fmt"
    "os"
    "os/exec"
)

func main() {
    fmt.Println("I want this captured")

    // I don't want this captured, but I want "ls" printed to console
    command := exec.Command("ls")
    // With these lines, the print statement and the ls statement are printed to console.
    // All are captured in a bash variable.
    command.Stdout = os.Stdout
    command.Stderr = os.Stderr
    
    command.Run()

}

Copy that into a file main.go and run it like this:

output=$(go run main.go)
echo "$output"

Thank you.

`output=$(my_cli_command)` _doesn't_ redirect stderr away from its usual destination in the first place. You don't need to change anything at all to get the behavior you describe. If you're seeing something different, we need a [mre] that lets us observe that ourselves. — Charles Duffy, May 24 '23 at 22:31
Your title mentions stderr, but the question refers to the output of the subprocess. Does the subprocess write to stdout or stderr? — Barmar, May 24 '23 at 22:33
Try it yourself: `my_cli_command() { echo "This is stdout"; echo "This is stderr" >&2; }; result=$(my_cli_command)`, and you'll see `This is stdout` stored in `result`, and `This is stderr` written to the terminal (assuming that the terminal was where you had stderr going in the first place). — Charles Duffy, May 24 '23 at 22:33
Are you asking how to get the subprocess's stdout connected to the CLI's stderr, so it won't be captured by the command substitution? — Barmar, May 24 '23 at 22:35
Charles: done, added an example in Go -- also, added quotes around the echo. Barmar: Yes, that's my question. The subprocess can be configured to write to stdout, stderr, or /dev/null, or any output stream. — Andre, May 24 '23 at 22:38
@Andre, note that `echo $output` is buggy -- it needs to be `echo "$output"` for the reasons given in [I just assigned a variable, but `echo $variable` shows something different](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) — Charles Duffy, May 24 '23 at 22:39

Charles Duffy · Accepted Answer · 2023-05-24T22:46:53.220

3

This isn't a bash problem, it's a Go problem. If you don't want the output of ls to be captured, connect its stdout to your Go program's stderr rather than its stdout.

package main

import (
    "fmt"
    "os"
    "os/exec"
)

func main() {
    fmt.Println("I want this captured")

    command := exec.Command("ls")
    command.Stdout = os.Stderr // not os.Stdout
    command.Stderr = os.Stderr
    command.Run()
}

edited May 24 '23 at 22:46

answered May 24 '23 at 22:43

Charles Duffy

280,126
43
390
441

Mark Reed · Answer 2 · 2023-05-25T17:44:47.547

To be clear, the ls output is captured because it goes to stdout. If you were to run a command that sent anything to stderr, it would be displayed on the terminal and not captured, just as you indicated you want. For example, if you change the Go code to do this:

command := exec.Command("ls" ,"/no/such/file")

You get this result:

 $ output=$(go run main.go)
 ls: /no/such/file: No such file or directory
 $ echo $output
 I want this captured

The solution for your case is to connect the standard output of the child command to os.Stderr instead of os.Stdout, as in Charles Duffy's answer.

Capturing main process stdout into bash variable, keep printing subprocess output to console?

2 Answers2