1

I'm trying to do multiple pipes with Go: 

ctags := exec.Command("ctags", "-x", "--c-types=f", "./tmp/"+fileName)
grep  := exec.Command("grep", "member")
awk   := exec.Command("awk", "'{$1=$2=$3=$4=\"\"; print $0}'")
grep.Stdin, _ = ctags.StdoutPipe()
awk.Stdin, _ = grep.StdoutPipe()
awk.Stdout = os.Stdout
_ = grep.Start()
_ = awk.Start()
_ = ctags.Run()
_ = grep.Wait()
_ = awk.Wait()

fmt.Println(awk)

This is what the expected output should look like:

$ ctags -x --c-types=f ./tmp/genetic_algorithm.py | grep member | awk '{$1=$2=$3=$4=""; print $0}'
    def __init__(self, vector, fitness_function):
    def __init__(self, population_size=10, crossover_points=[.3, .6],
    def apply_crossover(self, genotype_1, genotype_2):
    def apply_mutation(self, child_genotype):
    def calc_population_fitness(self, individuals):
    def evolve(self):
    def final_genotype(self):
    def fitness(self):
    def generate_offspring(self, selected):
    def genotype(self, indiv):
    def get_accuracy(nn_hidden_structure=[10]):
    def parse_bitstring(self, genotype):
    def run(self):
    def select_for_crossover(self):
    def slice_vector(self, vector):
    def validate_vector(self, vector, max_nodes=100):
    def vector_from_bitstring(self, genotype):

This is what I'm getting:

&{/usr/bin/awk [awk '{$1=$2=$3=$4=""; print $0}'] []  0xc4204ce020 0xc42007e008 <nil> [] <nil> 0xc4201fa900 exit status 2 <nil> <nil> true [0xc4204ce020 0xc42007e008 0xc4204ce048] [0xc4204ce048] [] [] 0xc420070360 <nil>}

I've found several examples of piping commands with Go, but most of them only pipe two commands. There's this example https://gist.github.com/dagoof/1477401, but when I arranged my code according to that logic, I got the same issue. Am I piping correctly? The command is just supposed to execute exuberant ctags on that Python file and then grep any rows indicating method and then only ignore the first four columns of the grepped output.

Soubriquet
  • 3,100
  • 10
  • 37
  • 52
  • Don't you want to print the output of the `awk` command instead of the value of a pointer to a `Cmd` struct? – e0k Dec 30 '16 at 18:44
  • @e0k Yeah, I want to capture the Stdout of awk. Working on that now. You can ignore that fmt.Println() if it's confusing. – Soubriquet Dec 30 '16 at 18:49
  • You may find it useful to redirect `Stderr` streams to `os.Stderr` so that you can see error messages: `awk.Stderr = os.Stderr`, etc. – e0k Dec 30 '16 at 19:01
  • @e0k Do you know how to grab os.Stdout and use it as a string? – Soubriquet Dec 30 '16 at 19:04
  • You probably want to read not from `os.Stdout` but the last [`StdoutPipe()`](https://golang.org/pkg/os/exec/#Cmd.StdoutPipe), which gives you an [`io.ReadCloser`](https://golang.org/pkg/io/#ReadCloser) (a `Reader` that is also a `Closer`). You could [`ReadFrom()`](https://golang.org/pkg/bytes/#Buffer.ReadFrom) this and store it in a `bytes.Buffer`. Then (if you really needed it as such), convert it to a `string`. But this is inefficient: see [From io.Reader to string in Go](http://stackoverflow.com/questions/9644139/from-io-reader-to-string-in-go). – e0k Dec 30 '16 at 19:23
  • @e0k That didn't work :/ buf := new(bytes.Buffer) buf.ReadFrom(awk.StdoutPipe()) fmt.Println(buf.String()) ./gitscrape.go:321: too many arguments in call to buf.ReadFrom – Soubriquet Dec 30 '16 at 19:35
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/131928/discussion-between-soubriquet-and-e0k). – Soubriquet Dec 30 '16 at 19:41

1 Answers1

2

The issue was with:

awk   := exec.Command("awk", "'{$1=$2=$3=$4=\"\"; print $0}'")

I just needed to remove the single quotes.

awk   := exec.Command("awk", "{$1=$2=$3=$4=\"\"; print $0}")
Soubriquet
  • 3,100
  • 10
  • 37
  • 52