Bash extracting value from pattern

Question

[{"id":"12345","name":"cat"},{"id":"67890","name":"dog"}]

Let's say I have the json array above, I want to use sed/grep to extract the id where I only know the name, for example "dog". I can't use json tool such as jq.

So do the processing where you have it or work on getting it (or some other proper tool) installed where you need to do the work. Almost every programming language has a library to work with json many even have it in the standard library. Doing this with tools like `sed` or `awk` is fragile and complicated. Do it right, you'll thank yourself later. — Etan Reisner, Apr 14 '15 at 15:41
@EtanReisner has the right idea. You still don't want to use **stream** processing tools to munge arbitrarily complex **tree** structures. — l0b0, Apr 14 '15 at 15:43
Always "I don't care that it's a bloody stupid idea, I want to use sed for this!" The server doesn't have jq preinstalled? You have a shell, you can execute from your home; compile your own jq and copy it over there. Even if that's somehow impossible, every server that's been set up in the last decade has Python, which has a standard library to rival PHP's. It has [JSON tools](https://docs.python.org/2/library/json.html). It's bad enough to treat everything like a nail if all you have is a hammer, but why would you stop looking for tools when he hammer is just the first thing you found? — Wintermute, Apr 14 '15 at 15:53
@sdharry I hope it didn't come across as personal; it wasn't meant that way. It's just that sometimes, when a few too many people in one day ask how to edit XML or JSON or (worst of all) HTML with sed and awk instead of tools that were actually made to handle these formats, it gets a little frustrating. — Wintermute, Apr 14 '15 at 16:14
@Wintermute the reason behind wanting to use sed/grep/whatever is this is modifying a pre-made shell installer that doesn't use any other languages (and we can't add another installers... to the installer). But I appreciate your comments. I'll mix in Python to get this working. — sdharry, Apr 14 '15 at 16:21

hek2mgl · Accepted Answer · 2015-04-14T22:03:57.737

Since the input is json data, I would use jq for this:

echo '[{"id":"12345","name":"cat"},{"id":"67890","name":"dog"}]' \
    | jq --raw-output '.[] | select(.name=="dog") | .id'

If it is not available on servers, install it. An alternative might be to use a programming language like python, ruby, perl, php.. If you want I can give an example in one of those languages. If they are all not available on your server, then the server installation does not fit your requirements.

As you requested it, here comes a trivial implementation in python:

import json

data='[{"id":"12345","name":"cat"},{"id":"67890","name":"dog"}]'
for item in json.loads(data):
    if item['name'] == "dog":
        print(item['id'])
        exit(0)

If you want to read from stdin as in the jq example use this:

import sys
import json

data=sys.stdin.read()
for item in json.loads(data):
    if item['name'] == "dog":
        print(item['id'])
        exit(0)

(The pythonists might blame me because the code is too verbose, I'm a python noob! ;))

If I read the question correctly, OP wanted `jq '.[] | select(.name == "dog").id'`. — Wintermute, Apr 14 '15 at 16:00
Thanks @hek2mgl this is really helpful - I would use jq but this will be mass distributed and I can't justify installation for the small task this does. I'll use Python. Can you provide an example? — sdharry, Apr 14 '15 at 16:05

score 0 · Answer 2 · answered Apr 14 '15 at 18:38

If you want to do this without jq, here is a solution with a mix of sed and grep:

sed 's/},{/}\n{/g' inputfile |grep '"name":"dog"' |sed 's/.*"id":"\([0-9]*\)".*/\1/g

The first part sed 's/},{/}\n{/g' inputfile put each couple on a separate line.

The second part grep '"name":"dog"' extracts the line with the id you are looking for (replace dog by the appropriate id)

The third part sed 's/.*"id":"\([0-9]*\)".*/\1/g extracts the value of id from the line.

Make sure you use gnu version of sed.

Hope that helps!

For this *limited* use case `sed 's/.*{"id":"\([0-9]\+\)","name":"dog".*/\1/'` would be enough. :) However, `sed` is not the right tool for that! — hek2mgl, Apr 14 '15 at 18:52

Bash extracting value from pattern

2 Answers2