1

This has been asked a million times and I know there are a million solns. However im restricted in that I cant install anything on this client server , so I have whatever bash can come up with :)

I'm referencing Parsing JSON with Unix tools and using this to read data and split into lines. 

$ cat demo.json
{"rows":[{"name":"server1.domain.com","Access":"Owner","version":"99","Business":"Owner1","Owner2":"Main_Apprve","Owner1":"","Owner2":"","BUS":"Marketing","type":"data","Egroup":["ALPHA","BETA","GAMA","DELTA"],"Ename":["D","U","G","T","V"],"stage":"TEST"}]}

However as you can see it splits the "Egroup" and others with multiple entries into single lines making it a little bit more difficult.

cat demo.json |     sed -e 's/[{}]/''/g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}>
    "rows":["name":"server1.domain.com"
    "Access":"Owner"
    "version":"99"
    "Business":"Owner1"
    "Owner2":"Main_Apprve"
    "Owner1":""
    "Owner2":""
    "BUS":"Marketing"
    "type":"data"
    "Egroup":["ALPHA"
    "BETA"
    "GAMA"
    "DELTA"]
    "Ename":["D"
    "U"
    "G"
    "T"
    "V"]
    "stage":"TEST"]

Im trying to capture the data so i can list using a shell script. How would you advise me to capture each variable and then reuse in reporting in a shell script?

 grep -Po '"Egroup":.*?[^\\]",' demo.json
"Egroup":["ALPHA",

As you can see this wouldn't work for lines with more than 1 entry.

Thoughts appreciated. ( btw Im open to python and perl options but without having to install any extra modules to use with json )

Community
  • 1
  • 1
Gripsiden
  • 467
  • 2
  • 15
  • Note - Ename has the same problem... ( I could look to capture between [ and ] somehow? ) – Gripsiden Oct 03 '16 at 08:31
  • If `JSON` is installed already, which is quite possible, it's very easy with Perl. – zdim Oct 03 '16 at 08:37
  • I need to try achieve with bash tools. I will need to roll this out to a lot of servers eventually and the client wants nothing else on the boxes.. – Gripsiden Oct 03 '16 at 08:39
  • 1
    Alright. I take it that your actual data isn't this simple, right? What you show can nicely be parsed with Perl, without modules. (I imagine that Perl is acceptable since you show sed+awk approach.) – zdim Oct 03 '16 at 08:48
  • Yes , thats correct. Have place some demo data for the purpose of display , so full of typos etc :) Im not a developer per se - so will need to figure it out in Perl. thanks – Gripsiden Oct 03 '16 at 08:51
  • OK -- if all else fails one can always parse it in Perl in a rather reasonable script (within a limit, after which we'd be reinventing modules). It'd be better if there are ready tools, that satisfy your restrictions. – zdim Oct 03 '16 at 08:55
  • can you post an example of the desired output? – I0_ol Oct 03 '16 at 11:37
  • 2
    Tell the client what he wants is unreasonable. `bash` is not suitable for parsing JSON. – chepner Oct 03 '16 at 11:53
  • 2
    If you go ahead and attempt this then you are likely to end up with a fragile solution that will keep breaking for different reasons that you hadn't anticipated. It is very simple to parse JSON with the correct tools, and very difficult and unwise to try to do it without. You need to go back to your customer and explain this to them as best you can. If you're "not a developer per se" then why are you taking this on at all? It would be far from simple even for someone who knows what they are doing. Bash just isn't up to the job. That's why we have Perl, and why Perl has modules. – Borodin Oct 03 '16 at 12:25

3 Answers3

0

It's simple using Python.

Example

$ python -c 'import sys, json; print json.load(sys.stdin)["rows"][0]["Egroup"]' <demo.json 
[u'ALPHA', u'BETA', u'GAMA', u'DELTA']
xxfelixxx
  • 6,512
  • 3
  • 31
  • 38
Juan Diego Godoy Robles
  • 14,447
  • 2
  • 38
  • 52
  • Hi , Yes , but however I dont have the luxury of the json module :) Traceback (most recent call last): File "", line 1, in ? ImportError: No module named json – Gripsiden Oct 03 '16 at 08:38
  • @Gripsiden What version of Python is installed on that system? The `json` module is a standard module since Python 2.6. There's an old 3rd party module for older systems, but I guess that's no good for your situation. – PM 2Ring Oct 03 '16 at 08:41
  • I tested on my own home server with 2.6.6 installed - $ python -c 'import sys, json; print json.load(sys.stdin)["rows"][0]["Egroup"]' – Gripsiden Oct 03 '16 at 08:49
0

Perl 1-liner using JSON module:

perl -lane 'use JSON; my $data = decode_json($_); print join( ",", @{ $data->{rows}->[0]->{Egroup} } )' demo.json

Output

ALPHA,BETA,GAMA,DELTA

If you do not have JSON installed, instead of trying to reinvent a JSON parser, you can copy the source of JSON::PP (PP means Pure Perl) and put it in your working directory:

/working/demo.json
/working/JSON/PP.pm   # <- put it here

perl -lane -I/working 'use JSON::PP; my $data = decode_json($_); print join( ",", @{ $data->{rows}->[0]->{Egroup} } )' demo.json
xxfelixxx
  • 6,512
  • 3
  • 31
  • 38
  • Thanks for your response. They dont have module "JSON.pm" – Gripsiden Oct 03 '16 at 10:43
  • If you go the non-JSON parser route, be warned that your regular expression may fail on edge cases, so you should look out for that, i.e. check that you actually matched what you are expecting. – xxfelixxx Oct 03 '16 at 10:56
  • @Gripsiden What version of Perl do you have? The JSON::PP module has been in core since 5.14.0. And of course you can always just bundle the source with your script as xxfelixxx suggests. – ThisSuitIsBlackNot Oct 03 '16 at 14:31
  • Sorry hadnt seen update. I will test with JSON PP. My perl version is v5.8.8. Thank you. – Gripsiden Oct 03 '16 at 14:49
0

I think it's good to add jq solution, because it's a great JSON tool:

<input jq --raw-output '.rows[].Egroup|join(",")'

You can see it in action here https://jqplay.org/s/de4X1TUBG4

aborruso
  • 4,938
  • 3
  • 23
  • 40