7

I want to GET some json data from a server. I do this using:

UPDATE=$(curl -i -H "Accept: application/json" -H "Content-Type: application/json" --cookie "${COOKIE_NAME}" "${1}/update/${DEVICE_NAME}");

Before this, the server is authenticated. The ${1} is the server domain, ${DEVICE_NAME} is the name of the device requesting the update.

This returns a JSON as follows:

[{"_id":"54ff35887d8ef574029b9166","user":"54fe4313883bcec2c0ac0d64","__v":0,"created":"2015-03-10T18:18:48.023Z","status":"available","pbo_udid":"lemaker","installation_script":"","description":"Prints hello world to console","package_name":"helloworld_1.0-1.deb","name":"Hello World V1"}]

I want to now do 2 things:

  1. Make sure data is returned (if no update is available, the server returns []
  2. Extract data, for instance package_name

How do I do these in Linux bash script?

Kousha
  • 32,871
  • 51
  • 172
  • 296
  • 3
    You will likely need to pass the JSON value to a CLI for a script processor that can deserialize the JSON string and extract the data you are looking for. (i.e. something like PERL, PHP, etc.). Of course, at that point, I would probably just write the entire logic to retrieve and extract the data in that scripting language. Outside of that, you are likely looking at putting together some awkward combination of `sed`, `awk`, etc. to, in essence, build a deserializer. – Mike Brant Mar 10 '15 at 18:55
  • There are some really great tools for parsing JSON from bash; `jq` is one of them. – Charles Duffy Mar 10 '15 at 20:33

2 Answers2

10

Much more easily and reliably done using jq or jsawk:

content=$(curl ...)
package_name=$(jq -r '.package_name' <<<"$content")
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
4

Assuming there's no nested array:

cat <<EOF | json_reformat | \
    sed -rne '/:/s@^\s+"(\w+)":\s+"([^"]+)",?@json_\1="\2"@gp'
[{"_id":"54ff35887d8ef574029b9166","user":"54fe4313883bcec2c0ac0d64","__v":0,"created":"2015-03-10T18:18:48.023Z","status":"available","pbo_udid":"lemaker","installation_script":"","description":"Prints hello world to console","package_name":"helloworld_1.0-1.deb","name":"Hello World V1"}]
EOF

returns

json__id="54ff35887d8ef574029b9166"
json_user="54fe4313883bcec2c0ac0d64"
json_created="2015-03-10T18:18:48.023Z"
json_status="available"
json_pbo_udid="lemaker"
json_description="Prints hello world to console"
json_package_name="helloworld_1.0-1.deb"
json_name="Hello World V1"

You need json_reformat for this to work.

EDIT : without json_reformat:

cat <<EOF | \
    sed -re 's@(\[|\]|\{|\})@@g' -e 's/,/\n/g' | \
    sed -re 's@"(\w+)":\s*"?([^"]*)"?@json_\1="\2"@g'
[{"_id":"54ff35887d8ef574029b9166","user":"54fe4313883bcec2c0ac0d64","__v":0,"created":"2015-03-10T18:18:48.023Z","status":"available","pbo_udid":"lemaker","installation_script":"","description":"Prints hello world to console","package_name":"helloworld_1.0-1.deb","name":"Hello World V1"}]
EOF

It returns (note the version number that is reformatted anyway):

json__id="54ff35887d8ef574029b9166"
json_user="54fe4313883bcec2c0ac0d64"
json___v="0"
json_created="2015-03-10T18:18:48.023Z"
json_status="available"
json_pbo_udid="lemaker"
json_installation_script=""
json_description="Prints hello world to console"
json_package_name="helloworld_1.0-1.deb"
json_name="Hello World V1"

You can now try parsing this text using eval or source it from stdin.

  • 1
    Thanks. Any way to use it without downloading `json_format`?? I don't want to install other files – Kousha Mar 10 '15 at 19:40
  • It might be possible to enhance the `sed` script. What I like in `json_reformat` is it spares you the burden of verifying and sanitizing the received data. –  Mar 10 '15 at 19:45
  • 1
    See my edited response, without `json_reformat`. –  Mar 10 '15 at 20:05
  • Pipe content retrieved from a web page into `eval`? Are you _trying_ to get your system rooted? – Charles Duffy Mar 10 '15 at 20:36
  • @charles-duffy Not mine, no... More seriously, this is a hint of course. Using the evil `eval` is certainly not the only option. There are few of them however as the OP wishes no supplemental package. –  Mar 10 '15 at 20:43
  • Quite true; one could redirect results so processed into a loop using `printf -v` to set contents into variables, or (safer) into as associative array, if using bash 4, but the sensitive work (of the actual parsing) remains something to which bash is unsuited without `json_reformat` or other external tools (and without auditing `json_reformat`, I'm not sure about the eval-safety of its output; using double-quotes is a smell on that count). – Charles Duffy Mar 10 '15 at 20:48