How to parse JSON with shell scripting on Linux?

Question

I have a Json output from which I need to extract a few parameters in linux.

This is the json output:

{
  "OwnerId":"121456789127",
  "ReservationId":"r-48465168",
  "Groups":[

  ],
  "Instances":[
    {
      "Monitoring":{
        "State":"disabled"
      },
      "PublicDnsName":null,
      "RootDeviceType":"ebs",
      "State":{
        "Code":16,
        "Name":"running"
      },
      "EbsOptimized":false,
      "LaunchTime":"2014-03-19T09:16:56.000Z",
      "PrivateIpAddress":"10.250.171.248",
      "ProductCodes":[
        {
          "ProductCodeId":"aacglxeowvn5hy8sznltowyqe",
          "ProductCodeType":"marketplace"
        }
      ],
      "VpcId":"vpc-86bab0e4",
      "StateTransitionReason":null,
      "InstanceId":"i-1234576",
      "ImageId":"ami-b7f6c5de",
      "PrivateDnsName":"ip-10-120-134-248.ec2.internal",
      "KeyName":"Test_Virginia",
      "SecurityGroups":[
        {
          "GroupName":"Test",
          "GroupId":"sg-12345b"
        }
      ],
      "ClientToken":"VYeFw1395220615808",
      "SubnetId":"subnet-12345314",
      "InstanceType":"t1.micro",
      "NetworkInterfaces":[
        {
          "Status":"in-use",
          "SourceDestCheck":true,
          "VpcId":"vpc-123456e4",
          "Description":"Primary network interface",
          "NetworkInterfaceId":"eni-3619f31d",
          "PrivateIpAddresses":[
            {
              "Primary":true,
              "PrivateIpAddress":"10.120.134.248"
            }
          ],
          "Attachment":{
            "Status":"attached",
            "DeviceIndex":0,
            "DeleteOnTermination":true,
            "AttachmentId":"eni-attach-9210dee8",
            "AttachTime":"2014-03-19T09:16:56.000Z"
          },
          "Groups":[
            {
              "GroupName":"Test",
              "GroupId":"sg-123456cb"
            }
          ],
          "SubnetId":"subnet-31236514",
          "OwnerId":"109030037527",
          "PrivateIpAddress":"10.120.134.248"
        }
      ],
      "SourceDestCheck":true,
      "Placement":{
        "Tenancy":"default",
        "GroupName":null,
        "AvailabilityZone":"us-east-1c"
      },
      "Hypervisor":"xen",
      "BlockDeviceMappings":[
        {
          "DeviceName":"/dev/sda",
          "Ebs":{
            "Status":"attached",
            "DeleteOnTermination":false,
            "VolumeId":"vol-37ff097b",
            "AttachTime":"2014-03-19T09:17:00.000Z"
          }
        }
      ],
      "Architecture":"x86_64",
      "KernelId":"aki-88aa75e1",
      "RootDeviceName":"/dev/sda1",
      "VirtualizationType":"paravirtual",
      "Tags":[
        {
          "Value":"Server for testing RDS feature in us-east-1c AZ",
          "Key":"Description"
        },
        {
          "Value":"RDS_Machine (us-east-1c)",
          "Key":"Name"
        },
          {
          "Value":"1234",
          "Key":"Cost.centre"
        },
        {
          "Value":"Jyoti Bhanot",
          "Key":"Owner"
        }
      ],
      "AmiLaunchIndex":0
    }
  ]
}

Expected output :

 Instance id         Name                           cost centre             Owner
    i-1234576          RDS_Machine (us-east-1c)        1234                   Jyoti Bhanot

I want to write a file that contains headings like instance id, tag like name, cost centre, owner. Below that, certain values from the json output. The output here given is just a example.

How can I do that using sed and awk?

Any lead is appreciated.

Thanks

Refer http://stackoverflow.com/questions/20488315/read-the-json-data-in-shell-script http://stackoverflow.com/questions/1955505/parsing-json-with-sed-and-awk — Rahul R Dhobi, Mar 27 '14 at 04:58
Why do you want to use `awk` for parsing JSON? You must use a dedicated json parsing tool like `jq` or `jsawk`. Below answer should work for you I believe. — anubhava, Mar 27 '14 at 06:52
ok for the sample above can you provide your expected output in question? — anubhava, Mar 27 '14 at 06:58
You're making same mistake. You expected output isn't matching with the input you provided. Input json has only one: `"InstanceId":"i-1234576"` — anubhava, Mar 27 '14 at 07:04
Still not right. `value 1.1` is in your expected output but not in the JSON — anubhava, Mar 27 '14 at 07:07
its pretty complicated JSON and difficult to figure out where the values are. I will try. — anubhava, Mar 27 '14 at 07:49
Hi Megha. For some reason, when I tested now, there's a JSON Parse error. Did you somehow modified your input JSON sample? — Muhammad Baja Aksha, Mar 27 '14 at 07:56
Sorry Megha I tried but this JSON structure is too cumbersome and I couldn't get all the values. — anubhava, Mar 28 '14 at 11:18

score 14 · Answer 1 · edited May 23 '17 at 11:45

14

Here's a sample using jsawk. Reference: Parsing JSON with Unix tools

Setup:

First download jsawk from https://github.com/micha/jsawk:

$ curl -L http://github.com/micha/jsawk/raw/master/jsawk > jsawk
$ chmod 755 jsawk && mv jsawk ~/bin/

You might want to install js-devel first before you can use jsawk. I'm using Fedora, so what I did was:

$ sudo yum install js-devel

The test:

I copied your JSON output sample to a text file. Called it sample.json. Here a sample to get a value from your JSON output sample:

$ jsawk 'return this.Instances[0].Monitoring.State' < sample.json
disabled
$ jsawk 'return this.Instances[0].VpcId' < sample.json
vpc-86bab0e4

For JSON data from a URL, you can use curl http://someserver.com/data.json instead of cat:

$ curl http://someserver.com/data.json | jsawk 'return this.Instances[0].VpcId'
vpc-86bab0e4

You can use these commands in your bash file to generate a new file that contains strings / text that you wanted. You can read more about jsawk from the GitHub link that I provided here.

Is this what you were looking for?

edited May 23 '17 at 11:45

Community

1
1

answered Mar 27 '14 at 05:03

Muhammad Baja Aksha

487
10
18

Looks like this is a duplicate from: http://stackoverflow.com/questions/22316714/bash-shell-extract-object-from-json-file?lq=1 and http://stackoverflow.com/questions/1955505/parsing-json-with-sed-and-awk – Muhammad Baja Aksha Mar 27 '14 at 05:07
2

as [Rahul R Dhobi](http://stackoverflow.com/users/3184706/rahul-r-dhobi) said, you might want to see this: http://stackoverflow.com/questions/1955505/parsing-json-with-sed-and-awk for your specific question. [jsawk](https://github.com/micha/jsawk) might be your answer. – Muhammad Baja Aksha Mar 27 '14 at 05:17
please see my edited answer. I won't be able to write you a shell script to generate the file that you asked, but you should get how to write it yourself from here. let me know if it works. – Muhammad Baja Aksha Mar 27 '14 at 06:20
1

You could write a script that calls this one for each of the 100s of servers. – icedwater Mar 27 '14 at 06:23
Refer to this: **[A quick guide to writing scripts using the bash shell](http://www.panix.com/~elflord/unix/bash-tute.html)** on how to write a shell script. Try it first on your own, then ask or search again for help. – Muhammad Baja Aksha Mar 27 '14 at 06:31

lirik90 · Answer 2 · 2023-04-17T03:22:56.927

Pure Bash 3.2+ without dependencies (such as jq, python, grep, etc.):

source <(curl -s -L -o- https://github.com/lirik90/bashJsonParser/raw/master/jsonParser.sh)
read -d '' JSON << EOF
// put your json here
EOF
JSON=$(minifyJson "$JSON")
id=$(parseJson "$JSON" Instances 0 InstanceId)
name=$(parseJson "$JSON" Instances 0 Tags 1 Value)
cost=$(parseJson "$JSON" Instances 0 Tags 2 Value)
owner=$(parseJson "$JSON" Instances 0 Tags 3 Value)
echo -e "Instance id |\t\t Name \t\t\t\t| cost centre | Owner"
echo -e "$id \t| $name  | $cost\t\t  | $owner"

Output:

Instance id |        Name               | cost centre | Owner
i-1234576   | RDS_Machine (us-east-1c)  | 1234        | Jyoti Bhanot

Try it.

How to parse JSON with shell scripting on Linux?

2 Answers2

Linked