0

I have program that lets users parse JSON data with jq. I am shelling out to jq rather than using a library because the ones I've found have weird and inconsistent behaviour. The problem I have is that jq often has | (pipe) characters and, that could potentially let users run non-jq commands. e.g.:

jq . | rm file.txt

How do I safely let users run this?

Right now, I invoke it as this in Go (where file.txt contains the raw json):

cmd = exec.Command("bash", "-c", fmt.Sprintf("cat file.txt | %s", cmd))

Thanks!

EDIT: As pointed out several times, this question isn't about piping commands together. It is about how to safely execute a jq command that has pipes in it. I don't want users to do this: curl 'api.icndb.com/jokes/random/3' | jq -r '.value[] | .joke | rm file.txt'. Please re-open.

user_78361084
  • 3,538
  • 22
  • 85
  • 147
  • 4
    You need to properly quote whatever you pass to jq. How exactly are you invoking `jq`? How are you handling the user string? Details matter. – William Pursell Oct 11 '20 at 04:11
  • Oh, ok. I've edited but if there is a better way please suggest! – user_78361084 Oct 11 '20 at 04:14
  • ok. The backticks can be replaced with double quotes in this case. I didn't tag as Go since the answer doesn't have to be Go...the part I'm interested in is the command line command I should run. – user_78361084 Oct 11 '20 at 04:18
  • Why can't you use the https://golang.org/pkg/encoding/json/ package from go standard library? – 1218985 Oct 11 '20 at 04:19
  • The users using the end service are familiar with jq. They need to extract the element(s) they want from the JSON. – user_78361084 Oct 11 '20 at 04:21
  • This causes problems: `curl 'http://api.icndb.com/jokes/random/3' | jq -r '.value[] | .joke'` – user_78361084 Oct 11 '20 at 04:22
  • I chose not to use https://github.com/itchyny/gojq because they don't dump the raw output of the jq command. Instead, you have to iterate through each of the jokes. This presents a problem since the jq command is not fixed and I have to handle a whole bunch of edge cases. – user_78361084 Oct 11 '20 at 04:25
  • 3
    The [`cat` is useless.](https://stackoverflow.com/questions/11710552/useless-use-of-cat) I guess you are using it as a placeholder, but if you can avoid invoking a shell at all, you can significantly simplify the problem. – tripleee Oct 11 '20 at 07:58
  • This question isn't about piping commands together. It is about how to safely execute a jq command that has pipes in it. I don't want users to do this: curl 'http://api.icndb.com/jokes/random/3' | jq -r '.value[] | .joke | rm file.txt' – user_78361084 Oct 11 '20 at 16:23

1 Answers1

3

You can use io.Pipe() which basically creates a synchronous in-memory pipe and it can be used to connect code expecting an io.Reader with code expecting an io.Writer. For instance, if you have a json content like this:

{
  "fruits": [
    "apples",
    "oranges",
    "pears"
  ]
}

You can try something like this:

package main

import (
    "bytes"
    "io"
    "os"
    "os/exec"
)

func main() {
    c1 := exec.Command("cat", "/home/fruits.json")
    c2 := exec.Command("jq", ".fruits[0:2]")

    r, w := io.Pipe()
    c1.Stdout = w
    c2.Stdin = r

    var b2 bytes.Buffer
    c2.Stdout = &b2

    c1.Start()
    c2.Start()
    c1.Wait()
    w.Close()
    c2.Wait()
    io.Copy(os.Stdout, &b2)
}

It will gives you something like this:

[
  "apples",
  "oranges"
]

Even though you can pipe several commands like this, I would strongly recommend to use the json package from standard library to Marshal and Unmarshal JSON.

1218985
  • 7,531
  • 2
  • 25
  • 31
  • The question was how to do it safely. This could potentially cause problems: curl 'http://api.icndb.com/jokes/random/3' | jq -r '.value[] | .joke | rm file.txt' – user_78361084 Oct 11 '20 at 16:22
  • @user_78361084, no, it can't (*), because no shell is involved in the above code. The argument to jq is only interpreted by jq itself, nothing else. * This is assuming you don't let the client control the filename. – Peter Oct 12 '20 at 12:30