0

I have object in JSON that has the structure as shown below and I want to convert it into a "fake 3D" associative BASH array with the following structure:

SHIPS[$mmsi:mmsi]=367513050
SHIPS[$mmsi:lat]=42.380329
...etc

In other words, I want the mmsi value combined with the key from each object to become the index to the associative array.

Right now, I am doing the following. The drawback is that it relies on the mmsi field to be the first field of each JSON object -- which is probably true but also not very reliable. This provides 3 objects: VESSEL_INDEX with all mmsis, KEY_INDEX with all existing keys, and VESSELS that can be accessed as VESSELS[$mmsi:$key]

EDIT: for clarity/completeness, my final goal is to:

  • declare an associated array
  • curl in a JSON file from a URL
  • update the associated array (with keys VESSELS[$mmsi:$key]) with the keys/values provided in the JSON object while leaving any unmentioned elements of the array unchanged.
    declare -A VESSELS
    declare -a VESSEL_INDEX
    declare -a KEY_INDEX

    while read -r keyvalue
    do
        key="${keyvalue%%=*}"
        value="${keyvalue#*=}"
        if [[ "$key" == "mmsi" ]]
        then
            mmsi="$value"
            [[ -n "$mmsi" ]] && VESSEL_INDEX+=("$mmsi")
        fi
        [[ ! " ${KEY_INDEX[*]} " =~ " ${key} " ]] && KEY_INDEX+=("${key}")
        [[ -n "$mmsi" ]] && VESSELS["$mmsi:$key"]="$value"

    done <<< "$(curl -sL "$AIS_URL/ships_full.json" | jq -r ".ships[]|to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]")"

with this input curled in:

{
  "count": 2
  "ships": [
    {
      "mmsi": 367513050,
      "lat": 42.380329,
      "lon": -71.042946,
      "distance": 0.317654,
      "mmsi_type": 1,
      "level": -28.161266,
      "count": 6895,
      "ppm": 4.918982,
      "heading": null,
      "cog": null,
      "speed": 0,
      "to_bow": 10,
      "to_stern": 17,
      "to_starboard": 3,
      "to_port": 5,
      "shiptype": 52,
      "msg_type": 0,
      "country": "US",
      "status": 0,
      "callsign": "WDG2188",
      "shipname": "VINCENT D. TIBBETTS",
      "destination": "BOSTON",
      "last_signal": 0
    },
    {
      "mmsi": 367447520,
      "lat": 42.324032,
      "lon": -70.994347,
      "distance": 3.777312,
      "mmsi_type": 1,
      "level": -37.657475,
      "count": 1103,
      "ppm": -1.157407,
      "heading": 156,
      "cog": 155.5,
      "speed": 28.4,
      "to_bow": 24,
      "to_stern": 20,
      "to_starboard": 9,
      "to_port": 4,
      "shiptype": 40,
      "msg_type": 0,
      "country": "US",
      "status": 0,
      "callsign": "WDF4062",
      "shipname": "SALACIA",
      "destination": "XX XXX>?? ???",
      "last_signal": 0
    },
  ],
  "error": false
}
kx1t
  • 13
  • 3

2 Answers2

0

Make jq construct a declare statement using brackets around the key and escaping with @sh:

unset SHIPS
declare -A SHIPS="($(jq -r '
  .ships[] | .mmsi as $mmsi
  | to_entries[] | @sh "[\("\($mmsi):\(.key)")]=\(.value)"
' ships_full.json ))"

Now you should be able to use the associative array $SHIPS in Bash:

echo "${SHIPS["367513050:lat"]}"

42.380329
pmf
  • 24,478
  • 2
  • 22
  • 31
  • Tnx - So I did run into an issue with your answer: it works fine if used exactly like you wrote - but it globs all of the output together in a single argument if you split the `declare` and assignment statements. (`unset SHIPS; declare -A SHIPS; SHIPS="($(jq -r ...` ) – kx1t Dec 27 '22 at 14:24
  • @kx1t Of course it fails, because `SHIPS="(['key']='value')"` is not how you declare arrays or assign item values in Bash. Please specify what you want to achieve. If you want one assignment per key-value pair, i.e. `SHIPS['key']='value'`, just rearrange the output of the jq filter accordingly. Though keep in mind that such an assignment is not a `declare` statement anymore, so you'd have to `eval` the output in order to be interpreted by Bash. – pmf Dec 27 '22 at 14:46
  • In the end, I want to curl in JSON (e.g., with `< <(curl -sSL https://kx1t.com/ais/ships.json)` ) and update an existing assoc array `SHIPS` with it. Any returned records will get updated from the JSON to `${SHIPS[$mmsi:$key]}` and any other records in the array will stay in place. As a result, I want to separately declare the associated array, and then later do a `SHIPS+=( ... )` – kx1t Dec 27 '22 at 15:21
  • @kx1t Regarding the input, just provide it on stdin the way you prefer it, e.g. `curl … | jq …` or `jq … <(curl …)` or the like. Regarding the array update, you basically want to read in a (new) array as described above, then [merge](https://stackoverflow.com/questions/29804909/) it with the already existing one. – pmf Dec 27 '22 at 15:30
  • Based on your second comment, I have this and it appears to work: `unset VESSELS; declare -A VESSELS; eval "$(jq -r '.ships[] | .mmsi as $mmsi | to_entries[] | @sh "VESSELS[\("\($mmsi):\(.key)")]=\(.value)"' < <(curl -sSL http://kx1t.com/ais/ships.json) )"; declare -p VESSELS` I will add it as a separate answer to my question and attribute you for it. Thanks again! – kx1t Dec 27 '22 at 15:40
0

Thanks to @pmf's answers and hints, I now have the following which appears to work. Posting it here so others can see it:

unset VESSELS
declare -A VESSELS

# ... other code here

eval "$(jq -r '.ships[] | .mmsi as $mmsi | to_entries[] | @sh "VESSELS[\("\($mmsi):\(.key)")]=\(.value)"' < <(curl -sSL http://xxxx.com/ais/ships.json) )"

# check that it worked:
declare -p VESSELS
kx1t
  • 13
  • 3