0

I am running a piece of JQ filter code. This is been run on a Windows Server 2016 with GIT BASH.

the code looks like the following: (don't mind the echo's and echo $i, this is for verbose reasons)

FILENAME="c:\Test\output4p.json"
for i in $(cat "$FILENAME" | jq -r .[].personnelNumber); do 
  echo $i
  echo
  jq '.[] | select(.personnelNumber=="'$i'")' "$FILENAME"
  echo
   done

it partly seems to run fine but sadly the output is. output1

so I on purpose remove the -r in the top JQ code to have a look what is actually going wrong.

$ "C:\TEST\echo.sh"
"3633014"

jq: error: syntax error, unexpected LITERAL, expecting ';' or ')' (Windows cmd shell quoting issues?) at <top-level>, line 1:
")                                 33014"
jq: 1 compile error



"3634594"

jq: error: syntax error, unexpected LITERAL, expecting ';' or ')' (Windows cmd shell quoting issues?) at <top-level>, line 1:
")                                 34594"
jq: 1 compile error



"3634708"

jq: error: syntax error, unexpected LITERAL, expecting ';' or ')' (Windows cmd shell quoting issues?) at <top-level>, line 1:
")                                 34708"
jq: 1 compile error



"3634847"

jq: error: syntax error, unexpected LITERAL, expecting ';' or ')' (Windows cmd shell quoting issues?) at <top-level>, line 1:
.[] | select(.personnelNumber==""3634847"")
jq: 1 compile error

if there is only 1 person in the json everything works as expected. but when there are more, the code only runs properly on the last person in the "For i in $cat" list.

someone else encounter something like this before?

thank you guys in advance

I am first making making a loop based upon the personnelNumber. Than for each personnelNumber I want to print the full array (which doesn't have an array ID) When this is working fine I'm going to take this code further into a curl command.

Jason Aller
  • 3,541
  • 28
  • 38
  • 38
Randy
  • 13
  • 2
  • 1
    If the only purpose of those exported numbers is to re-import them again for querying the same source, you should consider not leaving jq, and perform iteration and query in one go. – pmf Dec 20 '22 at 13:37
  • Thank you for you input, but the purpouse of this is that in between for eacht number a curl command with a specific API URL (inc. $i) is run. and in the JQ code im working on now should be used to fill the data section of that curl command with JSON information. – Randy Dec 20 '22 at 14:01
  • Then try out the command-line options `--arg` and `--argjson` (depending on your field's datatype), e.g. `jq --argjson i "$i" '.[] | select(.personnelNumber == $i)' "$FILENAME"` – pmf Dec 20 '22 at 14:22
  • that doesnt make a difference, the problem is that it seems to run incomplete code for the everybody except the last person. if i do a test with a curl request command for example, the first 3 lines are missing like 90 caracter positions (i check this by putting echo in front.) – Randy Dec 20 '22 at 14:35
  • 1
    That weirdly-formatted error message makes me suspect you've got DOS/Windows line endings somewhere, and are getting a (mostly-invisible) carriage return character in `$i`. This can cause [all sorts of weird problems](https://stackoverflow.com/questions/39527571). Replace `echo $i` with `echo "'$i'" | LC_ALL=C cat -vt`. If `$i` just has the number, that should print `'33014'`; if it has a carriage return, you'll get something like `'33014^M'`. – Gordon Davisson Dec 20 '22 at 17:06
  • @GordonDavisson your the man! exacly what was going on! i echoed the `echo "'$i'" | LC_ALL=C cat -vt` and ik got an output as `'3634594^M'` i starded to read you link and went to the Cygwin Section, added this line of code in the top of my code `set -o igncr` , now it runs as expected! – Randy Dec 21 '22 at 10:24

1 Answers1

1

The Comment of Gordon Davisson was exacly what was going on.

That weirdly-formatted error message makes me suspect you've got DOS/Windows line endings somewhere

Gordon passed me over a link to a earlyer post of something quite similar.

Are shell scripts sensitive to encoding and line endings?

I added the this piece of code to the top and i got all the correct outcommes.

set -o igncr

So conclusion, if you are running GITBASH or any Cygwin BASH make sure the Carriage Return in line endings are correct.

Randy
  • 13
  • 2