1

I am having a problem getting jq to work with the -r and --arg options to reference a variable and avoid "shell quoting" problems.

An existing topic here, suggests using --arg to define a command. I've tried that in this shell function that is designed to take log input and parse it as json if it is parseable as json. If it isn't it just dumps it out as a string.

function pparselogs () {
    while read data
    do
        jq --arg line "$data" -R -r '. as $line | try fromjson catch $line'
    done
}

The error that I keep running into is the following:

> echo "Hello World" | pparselogs .   
jq: error: syntax error, unexpected '|', expecting '$' or '[' or '{' (Unix shell quoting issues?) at <top-level>, line 1:
. as  | try fromjson catch
jq: 1 compile error

Given that the error is complaining about the |, it seems as though the "$data" arg value is not being picked up. But given the answer referenced above, this should be the way to do this. I can't see where I'm making a mistake.

Does anyone see the problem here?

fraxture
  • 5,113
  • 4
  • 43
  • 83
  • Can you provide some sample input data that can reproduce this error. – anubhava Dec 04 '19 at 16:05
  • 1
    Which shell are you using? – peak Dec 04 '19 at 16:06
  • @anubhava I added sample input. I get that error when just using a simple string. – fraxture Dec 04 '19 at 16:29
  • 4
    `. as $line | ...` isn't what you want, it means 'define a variable `$line` which contains what `.` refers to', but `.` is empty (no standard input). What you want is to change the value of `.` (since it is what `fromjson` will read) to the value of `$line`, which you can simply do by using `$line | ...`. That said using `--arg` needlessly complicate things, you'd better use the standard input as @anubhava did in his answer – Aaron Dec 04 '19 at 17:09
  • The `jq` error message (`. as | try from json catch`) implies you are using double quotes, not single quotes, to quote the filter; `$line` is being expanded as a shell argument before `jq` runs. – chepner Dec 05 '19 at 19:17
  • Using the code shown above, I don't get any errors (though I don't get any output, either). With `jq ... ". as $line | try from json catch $line"`, I get the syntax error you show. – chepner Dec 05 '19 at 19:19

1 Answers1

4

You may use this as shell function:

pparselogs() {
   while read data; do
      echo "$data" | jq -rR '. as $line | try fromjson catch $line'
   done
}

Then use it with invalid json:

echo 'Hello World' | pparselogs


Hello World

and with valid json:

echo '{"id":1, "message":"Hello World"}' | pparselogs


{
  "id": 1,
  "message": "Hello World"
}
anubhava
  • 761,203
  • 64
  • 569
  • 643