Go: receive os.cmd stdout and stderr in order

Question

I need to execute subcommand from go and process it stdout and stderr separately, with keeping order of ouput that comes to stdin/stdout. I've tried several differents ways, but could not achieve the correct order of output; following code shows that ouput handling order is absolutely random:

package main

import (
    "fmt"
    "log"
    "os/exec"
)

var (
    result = ""
)

type writer struct {
    result string
    write  func(bytes []byte)
}

func (writer *writer) Write(bytes []byte) (int, error) {
    writer.result += string(bytes) // process result later
    result += string(bytes)
    return len(bytes), nil
}

func main() {
    cmd := exec.Command("bash", "-c", "echo TEST1; echo TEST2 1>&2; echo TEST3")

    stderr := &writer{}
    cmd.Stderr = stderr

    stdout := &writer{}
    cmd.Stdout = stdout

    err := cmd.Start()
    if err != nil {
        log.Fatal(err)
    }

    err = cmd.Wait()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(result)
}

With several runs code can output following:

$ go run main.go
TEST1
TEST3
TEST2

I expect following result in all cases:

$ go run main.go
TEST1
TEST2
TEST3

I can not call cmd.CombinedOutput because I need to process stdout/stderr separately and in realtime.

ummm.. for some reason, i cannot reproduce your problem. I always get TEST1 TEST2 TEST3 — boaz_shuster, Aug 19 '15 at 10:13
Please look at this: http://stackoverflow.com/questions/4497817/save-stdout-stderr-and-stdoutstderr-synchronously — Alex Netkachov, Aug 19 '15 at 13:27
In general you can't since many operating systems will buffer standard output (especially if it's not connected to a terminal) but leave standard error unbuffered. I don't know of an OS agnostic way to change the buffering (e.g. FreeBSD has [`stdbuf`(1)](https://www.freebsd.org/cgi/man.cgi?query=stdbuf)). — Dave C, Aug 19 '15 at 16:23
I've tried to prepend commands with `stdbuf -o 0 -e 0` on Arch Linux and got the same results. Is there any way to tell OS do not to buffer results or to emulate terminal behaviour from go? OS specific way to do this will be okay. — Leonid Shagabutdinov, Aug 20 '15 at 11:47
Seems, there is really strange issue, because it is not working even in bash: `$ bash -c "echo TEST1; echo TEST2 1>&2; echo TEST3" > >(sed "s/^/out: /") 2> >(sed "s/^/err: /" >&2)` gives same random results; I give up... — Leonid Shagabutdinov, Aug 28 '15 at 22:10

score 2 · Answer 1 · answered Sep 17 '15 at 02:33

There is no "order" with the command that you're executing. They're parallel pipes, and as such they're effectively concurrent in the same way that two goroutines are concurrent. You can certainly store them in the order you receive them and tag them with their source by making use of channels or a mutex. In order to make the output not random with your synthetic example, you need to add a bit of a pause. I have used this method successfully with real commands, however:

package main

import (
    "fmt"
    "log"
    "os/exec"
    "sync"
)

var (
    result = ""
)

type write struct {
    source string
    data   string
}

type writer struct {
    source string

    mu     *sync.Mutex
    writes *[]write
}

func (w *writer) Write(bytes []byte) (int, error) {
    w.mu.Lock()
    defer w.mu.Unlock()
    *w.writes = append(*w.writes, write{
            source: w.source,
            data:   string(bytes),
    })
    return len(bytes), nil
}

func main() {
    cmd := exec.Command("bash", "-c", "echo TEST1; sleep .1; echo TEST2 1>&2; sleep .1; echo TEST3")

    var mu sync.Mutex
    var writes []write

    cmd.Stderr = &writer{
            source: "STDERR",
            mu:     &mu,
            writes: &writes,
    }
    cmd.Stdout = &writer{
            source: "STDOUT",
            mu:     &mu,
            writes: &writes,
    }

    err := cmd.Start()
    if err != nil {
            log.Fatal(err)
    }

    err = cmd.Wait()
    if err != nil {
            log.Fatal(err)
    }

    fmt.Printf("%q\n", writes)
}

will produce

[{"STDOUT" "TEST1\n"} {"STDERR" "TEST2\n"} {"STDOUT" "TEST3\n"}]

The problem is that stdout and stderr can produce one program and there is no way to setup pause between stdout and stderr message in that program; e.g. instead of `"bash", "-c"` could be another program that produce both of stdout and stderr. — Leonid Shagabutdinov, Sep 17 '15 at 07:55
@Leo Standard output and error are like pipes of water flowing from your subcommand to your program. If you put something in both pipes at nearly the same time, there is no guarantee that they will reach you in the same order. The solution above is as good as you can get without modifying the target, as it stores them in the order you receive them. You need to handle the case where things are slightly out of order in your program. — Kyle Lemons, Sep 19 '15 at 17:02
Therefore mutex locking provide much better ordering than without locking. Order was wrong only three times out of twenty tries of my machine, so I think it is best that it is possible to do with pipes... I just wondering how can default go implementation of CombinedOutput works (http://golang.org/src/os/exec/exec.go?s=10901:10947#L404), because they just assign bytes.Buffer that should behave exactly as writer with mutex. Am I missing something? — Leonid Shagabutdinov, Sep 21 '15 at 15:48

Go: receive os.cmd stdout and stderr in order

1 Answers1