Nested while loop with input from main while in shell script

Question

I am having Json file and i am trying to parse it by using below

#!/bin/ksh
while read rec
 do
    while read line
    do
        firstname=`echo $line | sed -n -e 's/^.*\(full-name\)/\1/p' | cut -f3 -d'"'`
        id=`echo $line | sed -n -e 's/^.*\(id\)/\1/p' | cut -f3 -d'"'`
        echo "${firstname}'|'${id}"
        done < `echo $rec | nawk 'gsub("}}}}", "\n")' | sed 's/{"results"//g'`

done < /var/tmp/Cloud_test.txt

My sample file is :

{"results":[{"general-info":{"full-name":"TELOS MANAGEMENT","body":{"party":{"xrefs":{"xref":[{"id":"66666"}]}}}},"_id":"91002551"},{"_id":"222222","body":{"party":{"general-info":{"full-name":"DO REUSE"},"xrefs":{"xref":[{"id":"777777"}]}}}}]}

Expected Result:
TELOS MANAGEMENT|66666
DO REUSE|777777

I am facing problem in inside while passing parameter. Its not getting passed line by line. Its passed complete line and result is not coming as expected. Please help to get it fixed.

`jq` is the right tool for this sort of job. [Parsing any sort of nested file structure in Bash is a Bad Idea™.](https://mywiki.wooledge.org/BashWeaknesses) — l0b0, Apr 27 '18 at 03:18

score 0 · Answer 1 · edited Apr 27 '18 at 07:26

0

This is a more complicated (double) case of this question.

The following works for me:

cat sample.json | sed -e 's/"full-name"/\n&/g' | tail -n+2 | sed -e 's/"full-name":"\([^"]*\).*{"id":"\([^"]*\).*/\1\|\2/'

edited Apr 27 '18 at 07:26

peak

105,803
17
152
177

answered Apr 27 '18 at 04:45

Jens Petersen

166
9

tail -n+2 is not working for me. Could you please confirm anything wrong with this? – user7392087 Apr 27 '18 at 13:07
You could replace `tail -n+2` with `grep -v '{"results":'`, or check the manpage for `tail`. – Jens Petersen Apr 27 '18 at 13:25

score 0 · Answer 2 · answered Apr 27 '18 at 07:24

As pointed out by @l0b0 this kind of problem is best solved using a JSON-aware tool such as jq. Here, then, is a jq solution.

It must be pointed out, however, that the sample input is strangely irregular, so the requirements are not so clear. If the JSON were more regular, the jq solution would be simpler.

In any case, the following jq filter does produce the result as described:

.results[]
| ..
| objects
| select(has("general-info"))
| [(.["general-info"]|.["full-name"]), (.. | .id? // empty)]
| join("|")

Simplification

The second last line above could be simplified to:

[."general-info"."full-name", (.. | .id? // empty)]

Thats correct. File is not in same sequence order. That is the issue. We heard that json can be out of sequnce {"results": [{"general-info":{"full-name":"TELOS MANAGEMENT","body":{"party":{"xrefs":{"xref":[{"id":"66666"}]}}}},"_id":"91002551"}, {"_id":"222222","body":{"party":{"general-info":{"full-name":"DO REUSE"},"xrefs":{"xref":[{"id":"777777"}]}}}}, {"_id":"333333","body":{"party":{"general-info":{"full-name":"AAAA MANAGEMENT"},"xrefs":{"xref":[{"id":"888888"}]}}}},]} — user7392087, Apr 27 '18 at 13:10

Nested while loop with input from main while in shell script

2 Answers2

Simplification