0

I have this shell script here that will go through each line in a json response and fetch me all the siteIds in a single comma delimited String.

input='[
  {
    "siteIds": "93970,168794,34583,96844,34343,29095,98786,136728,33441,35913",
    "status": "Failed for some reason"
  },
  {
    "siteIds": "28709,29810,84178,31027,29985,100487,35486,36068,37001,36360",
    "status": "Failed for reason xyz"
  },
  {
    "siteIds": "190873,28714,93455,35995,12542,77957,102080,32006,109115,28902",
    "status": "Success"
  },
  {
    "siteIds": "30514,33349,29178,77989,30460,35541,36765,29637,97585,29263",
    "status": "Failed:..."
  },
  {
    "siteIds": "36913,36741,30435,35007,109238,30606,33669,76823,75882,30265",
    "status": "Success"
  }
]'

siteIds=""
while read -r line; do
  if [[ $line = *"status\": \"Failed"* ]]; then
    echo $line
    siteId=$(echo $line | grep -o '"siteIds": "[^"]*"' | awk -F': ' '{print $2}' | tr -d '"')
    siteIds="$siteIds$siteId,"
  fi
done <<< "$input"

if [[ -z "$siteIds" ]]; then
    siteIds = "No failed siteIds found."
fi

echo "${siteIds%?}"
echo "finished"

This is the response I am getting:

"status": "Failed for some reason"
"status": "Failed for reason xyz"
"status": "Failed:..."
,,
finished

How do I get the response to be in this format?

93970,168794,34583,96844,34343,29095,98786,136728,33441,35913,28709,29810,84178,31027,29985,100487,35486,36068,37001,36360,30514,33349,29178,77989,30460,35541,36765,29637,97585,29263

Seems like it is getting the Failed statuses but not able to append the siteIds correctly. I'm not experienced in shell scripts so any help would be appreciated!

Gojo
  • 151
  • 7
  • 2
    Install `jq`, or Python, or _something_ with a real JSON parser. Trying to use generic tools that don't know the JSON format to parse JSON is extremely error-prone: You can get it to look like it works with one input file but then have it fail the next day when your input file has slightly different whitespace. – Charles Duffy Feb 07 '23 at 20:22
  • 1
    Don't go down that road at all: use tools written with the specification in mind, and everyone will be a lot happier (including any business partners providing the files you're trying to parse; when I maintained an API my worst enemies were people using hand-rolled parsers, because they'd come up with bugs that were caused by their own code not handling content that was perfectly specification-compliant). – Charles Duffy Feb 07 '23 at 20:25
  • In this case, `jq -r '.[] | select(.status | test("failed.*";"i")) | .siteIds'`should be basically what you're looking for. – Charles Duffy Feb 07 '23 at 20:27
  • And on that note, complain to whoever is providing this input for giving you comma-separated streams of site IDs instead of proper arrays. – chepner Feb 07 '23 at 20:28
  • (maybe `jq -r '[ .[] | select(.status | test("failed.*";"i")) | .siteIds ] | join(",")'` to get all your output onto one line) – Charles Duffy Feb 07 '23 at 20:39
  • I think using a python script would be the best way about it, however we ended up changing the api response itself to concatenate failed ids and successful ids. thanks all!! – Gojo Feb 10 '23 at 00:41

0 Answers0