Why does `inputs` skip the first line of the input file?

Question

Whenever I provide a file to jq and use the inputs command, I only get back all lines but the first one and I'm wondering why is that

I'm currently using jq 1.6 and I'm trying to use the inputs command to convert a TSV (Tab Separated Values) into JSON by grabbing the first line of the file as the headers and placing each header as the key of the corresponding value in the next lines

If I execute the following

echo -n 'line1'$'\n''line2' | jq -R 'inputs'

The result is

line2

and not

line1
line2

as I'd expect

As a workaround I'm currently prepending a new line to the input i'm giving to jq as in

echo -n $'\n''line1'$'\n''line2' | jq -R 'inputs'

but my expectation is to be able to use inputs and have it process the first line

chepner · Accepted Answer · 2019-05-05T20:33:49.070

jq itself is reading the first line, then inputs (having received that line as its input) will read the rest. Typically, you want to use the -n option to prevent jq from doing any reading itself, letting any input or inputs filters do the actual reading.

$ echo -n $'line1\nline2\n' | jq -nR 'inputs'
"line1"
"line2"

In your case, letting jq read the header and inputs read the rest of the data is reasonable, but you have to do something with the header. This is probably more complicated than it needs to be, but does the job:

$ cat tmp.tsv
foo bar baz
1   2   3
4   5   6
$ jq -R 'split("\t") as $h | [inputs | split("\t") | [{key: $h[0], value: .[0]}, {key: $h[1], value: .[1]}, {key: $h[2], value: .[2]}] | from_entries]' tmp.tsv
[
  {
    "foo": "1",
    "bar": "2",
    "baz": "3"
  },
  {
    "foo": "4",
    "bar": "5",
    "baz": "6"
  }
]

jq reads the first line and splits it into the array h, then feeds that line to a filter which ignores it but uses inputs to read the rest of the lines, splitting each one up and creating a dict using (repeatedly) the value of $h.

It would probably be better to encourage the use of `objectify` defined e.g. as `def objectify($h): . as $in | reduce range(0; $h|length) as $i ({}; .[$h[$i]] = $in[$i]);` — peak, May 06 '19 at 04:06

peak · Answer 2 · 2019-05-06T16:09:18.480

@chepner's explanation of -n is fine, but I'd like to illustrate how debug can help clarify and/or demystify jq's behavior.

Since every jq program is a filter, it is always possible to see what the input to any given filter is by prepending it with debug, so e.g.:

echo 1 2 3 | jq 'debug | inputs'

yields:

["DEBUG:",1]
2
3

That is, in this case, the input to inputs is 1, which is consumed by inputs and otherwise ignored.

We can similarly examine the effect of using the -n command-line option:

echo 1 2 3 | jq -n 'debug | inputs'
["DEBUG:",null]
1
2
3

That is, we can now see that the -n option tells jq to provide null as the input to inputs, rather than obtaining it from STDIN.

In short, debug is helpful not only for debugging, but also for DIY understanding.

Why does `inputs` skip the first line of the input file?

2 Answers2

Linked