238

I can't seem to get jq to behave "normally" in a shell pipeline. For example:

$ curl -s https://api.github.com/users/octocat/repos | jq | cat

results in jq simply printing out its help text*. The same thing happens if I try to redirect jq's output to a file:

$ curl -s https://api.github.com/users/octocat/repos | jq > /tmp/stuff.json

Is jq deliberately bailing out if it determines that it's not being run from a tty? How can I prevent this behavior so that I can use jq in a pipeline?

Edit: it looks like this is no longer an issue in recent versions of jq. I have jq-1.6 now and the examples above work as expected.

* (I realize this example contains a useless use of cat; it's for illustration purposes only)

mgalgs
  • 15,671
  • 11
  • 61
  • 74

2 Answers2

391

You need to supply a filter as an argument. To pass the JSON through unmodified other than the pretty printing jq provides by default, use the identity filter .:

curl -s https://api.github.com/users/octocat/repos | jq '.' | cat
chepner
  • 497,756
  • 71
  • 530
  • 681
  • 1
    Why the pipe to "cat" ? Is it really necessary ? – Renan Ranelli Oct 21 '15 at 03:30
  • 26
    I left it in because the OP had it; I assume it's just a placeholder to indicate that `jq` is both reading from a pipe and writing to another pipe. If the desire is to simply see the output of `jq`, then `cat` itself is unnecessary. – chepner Oct 21 '15 at 03:36
  • 4
    (1) jq version 1.5 has been enhanced so that the . is unnecessary in cases such as this. (2) There is no need to quote the ".". – peak Oct 23 '15 at 03:27
  • 9
    @peak It may not be necessary for a pipeline, but I have 1.5 and had to provide the "." to redirect the output to a file. – MHarris Apr 05 '16 at 14:15
  • 4
    I quote the `.` out of habit, since my standard practice is to first see what the JSON looks like, then go back and start adding to the filter, which more often than not will need to be quoted. – chepner Apr 05 '16 at 14:16
  • 1
    @peak I can verify that on OSX at least jq 1.5 requires the . for it to output to a pipe. – whaleberg Jul 28 '17 at 16:18
  • 1
    @whaleberg - Sorry for not being clearer. I meant that an enhancement was made to jq 1.5 (i.e. after the release of jq 1.5). In any case, I'd recommend always using an explicit filter. – peak Jul 28 '17 at 17:18
  • actually the quotes around the dot are not necessary, `jq .` is fine – Walter Tross Sep 14 '17 at 22:27
  • 11
    @WalterTross Yes, and I explained in a comment 18 months ago why I quote it. – chepner Sep 14 '17 at 23:38
  • I know, I just added my comment for faster&clearer reference. Not everyone has the time to read comments carefully – Walter Tross Sep 16 '17 at 11:43
18

One use case I have found myself doing frequently as well is "How do I construct JSON data to supply into other shell commands, for example curl?" The way I do this is by using the --null-input/-n option:

Don’t read any input at all! Instead, the filter is run once using null as the input. This is useful when using jq as a simple calculator or to construct JSON data from scratch.

And an example passing it into curl:

jq -n '{key: "value"}' | curl -d @- \
  --url 'https://some.url.com' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json'
mkobit
  • 43,979
  • 12
  • 156
  • 150
  • 15
    That's cool, but not sure how it's relevant to this question? – mgalgs May 20 '17 at 17:51
  • 5
    @mgalgs I found myself wanting to generate/use `jq` at the start of a shell pipeline rather than in the middle/end for filtering down on some data. The `curl` example is a basic one, but I found myself frequently hand typing in JSON data to `curl` and trying to get quoting right, so I thought it might be useful to others as well. – mkobit May 20 '17 at 19:08
  • The question title also being "How to use `jq` in a shell pipeline?" and this being a highly voted question also leads towards this being relevant for those who stumble upon it. – mkobit Mar 03 '23 at 02:12