27

I'd like to extract the "id" key from this single line of JSON.

I believe this can be accomplished with grep, but I am not sure on the correct way.

If there is a better way that does not have dependencies, I would be interested.

Here is my example output:

{
  "data": {
    "name": "test",
    "id": "4dCYd4W9i6gHQHvd",
    "domains": ["www.test.domain.com", "test.domain.com"],
    "serverid": "bbBdbbHF8PajW221",
    "ssl": null,
    "runtime": "php5.6",
    "sysuserid": "4gm4K3lUerbSPfxz",
    "datecreated": 1474597357
  },
  "actionid": "WXVAAHQDCSILMYTV"
}
Sven Märki
  • 189
  • 1
  • 12
petebocken
  • 395
  • 1
  • 3
  • 8
  • This may help you: http://stackoverflow.com/questions/1955505/parsing-json-with-unix-tools – deathyr Sep 23 '16 at 03:12
  • 1
    try `grep -o -P '"id":.*?",' json.file |sed 's/"\|,//g'` , should help you ..but it would be better if you would go for some json aware tools. – P.... Sep 23 '16 at 04:23
  • @BenjaminW.: Shouldn't the `data:`'s contents be enclosed within `[ ]` ? – Inian Sep 23 '16 at 05:42
  • You can look for some Q&A : http://stackoverflow.com/questions/1955505/parsing-json-with-unix-tools, http://stackoverflow.com/questions/20488315/read-the-json-data-in-shell-script, http://unix.stackexchange.com/questions/121718/how-to-parse-json-with-shell-scripting-in-linux – dkb Sep 23 '16 at 05:43

5 Answers5

70

If you have a grep that can do Perl compatible regular expressions (PCRE):

$ grep -Po '"id": *\K"[^"]*"' infile.json
"4dCYd4W9i6gHQHvd"
  • -P enables PCRE
  • -o retains nothing but the match
  • "id": * matches "id" and an arbitrary amount of spaces
  • \K throws away everything to its left ("variable size positive look-behind")
  • "[^"]*" matches two quotes and all the non-quotes between them

If your grep can't do that, you an use

$ grep -o '"id": *"[^"]*"' infile.json | grep -o '"[^"]*"$'
"4dCYd4W9i6gHQHvd"

This uses grep twice. The result of the first command is "id": "4dCYd4W9i6gHQHvd"; the second command removes everything but a pair of quotes and the non-quotes between them, anchored at the end of the string ($).

But, as pointed out, you shouldn't use grep for this, but a tool that can parse JSON – for example jq:

$ jq '.data.id' infile.json
"4dCYd4W9i6gHQHvd"

This is just a simple filter for the id key in the data object. To get rid of the double quotes, you can use the -r ("raw output") option:

$ jq -r '.data.id' infile.json
4dCYd4W9i6gHQHvd

jq can also neatly pretty print your JSON:

$ jq . infile.json
{
  "data": {
    "name": "test",
    "id": "4dCYd4W9i6gHQHvd",
    "domains": [
      "www.test.domain.com",
      "test.domain.com"
    ],
    "serverid": "bbBdbbHF8PajW221",
    "ssl": null,
    "runtime": "php5.6",
    "sysuserid": "4gm4K3lUerbSPfxz",
    "datecreated": 1474597357
  },
  "actionid": "WXVAAHQDCSILMYTV"
}
Benjamin W.
  • 46,058
  • 19
  • 106
  • 116
  • 2
    @BenjaminW. Thanks! Using grep will work perfect for this situation. This works `grep -o '"id": *"[^"]*"' test.txt | grep -o '"[^"]*"$'` How would I remove the quotes as well, leaving only the ID? – petebocken Sep 23 '16 at 13:11
  • 3
    @petebocken You'd have to remove the last quote from the first command and the first and last quote from the second one: `grep -o '"id": *"[^"]*' test.txt | grep -o '[^"]*$'` – Benjamin W. Sep 23 '16 at 13:28
  • 1
    @BenjaminW. I tried several edits and for some reason couldn't find the right one to remove. This works perfect, thanks! – petebocken Sep 23 '16 at 13:30
  • @BenjaminW. `'"id": *\K"[^"]*"'` this works for me, but how can i make sure i remove the double quotes from the result. – Salil Lambay Jul 22 '20 at 17:30
  • @SalilLambay You should use `jq -r` instead of grep, or you have to pipe to another instance of grep. – Benjamin W. Jul 22 '20 at 19:02
  • 1
    @BenjaminW. I tried the 2 grep solution and it worked :) cheers – Salil Lambay Jul 22 '20 at 19:03
  • @BenjaminW. This did what I needed without the quotes in the return, and making sure that escaped quotes are handled `"id": *"\K(?:\\"|[^"]))`. Still would need to handle escapes after you get it back, likely with `sed`. Play with my example: https://regex101.com/r/Lc9k5p/1 – VertigoRay Oct 18 '20 at 20:26
  • @BenjaminW. Sadly this needed to be done without installing more software on the systems, and a more robust programming language, like python, wouldn't be consistently available. – VertigoRay Oct 18 '20 at 22:12
  • Thanks a lot @BenjaminW. This grep porn saved my several hours of manual effort. Works well with Mac Catalina's native grep! – BhaveshDiwan Jun 17 '21 at 12:31
10

Just pipe your data to jq and select by keys

"data": {
    "name": "test",
    "id": "4dCYd4W9i6gHQHvd",
    "domains": [
      "www.test.domain.com",
      "test.domain.com"
    ],
    "serverid": "bbBdbbHF8PajW221",
    "ssl": null,
    "runtime": "php5.6",
    "sysuserid": "4gm4K3lUerbSPfxz",
    "datecreated": 1474597357
  },
  "actionid": "WXVAAHQDCSILMYTV"
} | jq '.data.id'     

# 4dCYd4W9i6gHQHvd

Tutorial Here

jasonleonhard
  • 12,047
  • 89
  • 66
7

I found myself that the best way is to use python, as it handles JSON natively and is preinstalled on most systems these days, unlike jq:

$ python -c 'import sys, json; print(json.load(sys.stdin)["data"]["id"])' < infile.json
4dCYd4W9i6gHQHvd
Kamil Christ
  • 324
  • 4
  • 4
  • 2
    Liking this answer because it uses what is there. It's all too easy to just go installing more and more to solve things like this. Very simple, but overlooked approach! – kmjb May 22 '20 at 16:56
  • best and easy solution – Vovan Mar 17 '22 at 13:12
  • This can also be piped from `curl` request like so: `curl https://jsonplaceholder.typicode.com/todos/1 | python -c 'import sys, json; print(json.load(sys.stdin)["title"])'` and it works for both python v2 & v3. – Ibrahim.H Mar 13 '23 at 07:25
5

No python ,jq, awk, sed just GNU grep:

#!/bin/bash
json='{"data": {"name": "test", "id": "4dCYd4W9i6gHQHvd", "domains": ["www.test.domain.com", "test.domain.com"], "serverid": "bbBdbbHF8PajW221", "ssl": null, "runtime": "php5.6", "sysuserid": "4gm4K3lUerbSPfxz", "datecreated": 1474597357}, "actionid": "WXVAAHQDCSILMYTV"}' 
        
echo $json | grep -o '"id": "[^"]*' | grep -o '[^"]*$'

Tested & working here: https://ideone.com/EG7fv7

source: https://brianchildress.co/parse-json-using-grep

social
  • 329
  • 3
  • 8
  • It works, however the only issue comes when piping the output of `curl` to `grep`, since `curl` uses the stderr instead of stdout to output the results, we must pipe from the stderr (using `2>&1`) like so:`curl https://jsonplaceholder.typicode.com/todos/1 2>&1 | grep -o '"title": "[^"]*' | grep -o '[^"]*$'`. – Ibrahim.H Mar 13 '23 at 07:43
0

$ grep -oP '"id": *"\K[^"]*' infile.json

4dCYd4W9i6gHQHvd

Hopefully it will work for all. As this will work for me to print without quotes.

Akshat Pareek
  • 1