Parsing JSON files in bash?

Question

I have some API result I wanted to clean up. I have been trying with grep to only show the Name under Technologies but I can't seem to figure it out yet.

Any ideas? (Here's a small snippet of the giant json file)

    {
  "Results": [
    {
      "Lookup": null,
      "Result": {
        "Paths": [
          {
            "Domain": "builtwith.com",
            "Url": "",
            "SubDomain": "",
            "Technologies": [
              {
                "Categories": null,
                "Name": "Amazon CloudFront",
                "Tag": "cdns",
                "FirstDetected": 1386284400000,
                "LastDetected": 1411599600000
              },
              {
                "Categories": null,
                "Name": "ASP.NET 4.0",
                "Tag": "framework",
                "FirstDetected": 1385679600000,
                "LastDetected": 1430265600000
              },
              {
                "Categories": null,
                "Name": "Comodo PositiveSSL",
                "Tag": "ssl",
                "FirstDetected": 1372806000000,
                "LastDetected": 1430265600000
              }
           ]

I am not stuck on grep.. but it seems like grep would the best idea. I am open to others if needed.

BTW, personally, I tend to agree with the answer here (jq) over the currently-highest-voted answer on the question this is duplicative of (jsawk). jq *is* given as an answer on that question, however. — Charles Duffy, Apr 30 '15 at 22:05

John Kugelman · Answer 1 · 2015-04-30T22:05:01.030

If you're parsing JSON in bash, I insist that you use jq.

jq is like sed for JSON data – you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

Check out what it can do:

$ jq -r '.Results[].Result.Paths[].Technologies[].Name' results.json 
Amazon CloudFront
ASP.NET 4.0
Comodo PositiveSSL

$ cat results.json
{
 "Results": [
    {
      "Lookup": null,
      "Result": {
        "Paths": [
          {
            "Domain": "builtwith.com",
            "Url": "",
            "SubDomain": "",
            "Technologies": [
              {
                "Categories": null,
                "Name": "Amazon CloudFront",
                "Tag": "cdns",
                "FirstDetected": 1386284400000,
                "LastDetected": 1411599600000
              },
              {
                "Categories": null,
                "Name": "ASP.NET 4.0",
                "Tag": "framework",
                "FirstDetected": 1385679600000,
                "LastDetected": 1430265600000
              },
              {
                "Categories": null,
                "Name": "Comodo PositiveSSL",
                "Tag": "ssl",
                "FirstDetected": 1372806000000,
                "LastDetected": 1430265600000
              }
            ]
          }
        ]
      }
    }
  ]
}

WOW! That is super awesome! I have been looking for that for a week now!! — user3610137, Apr 30 '15 at 22:02
@user3610137, it's really hard to help with a "permission denied" without more info. That could be anything from not having +x on the directory it's in to trying to run it from a home directory mounted `noexec` to SELinux policies prohibiting execution of binaries not flagged as installed by system packages to... well, who knows how many other things that could be? — Charles Duffy, Apr 30 '15 at 22:09
I figured it out. Sorry about that. Awesome tool! Thank you again, John Kugelman! — user3610137, May 02 '15 at 20:34

Parsing JSON files in bash?

1 Answers1