How to use(read) value from associative array during jq map?

Question

I am reading aws ssm parameters via jq with json, and want to put them into a yaml file after some processing and alias mapping.

I have one associative array of mapping parameter name to some internal name (see process.sh)

My simplified example.json is

{
  "Parameters": [
    {
      "Name": "/dev/applications/web/some_great_key",
      "Value": "magicvaluenotrelevant"
    },
    {
      "Name": "/dev/applications/web/api_magic",
      "Value": "blabla"
    }
  ]
}

My bash script process.sh is

#!/usr/bin/env bash

declare -A ssmMap
ssmMap[/dev/applications/web/some_great_key]=theGreatKEYAlias
ssmMap[/dev/applications/web/api_magic]=apiKey

jq -r '
  .Parameters
  | map({
        Name: .Name,
        parameterValue: .Value
    })
  | map((${!ssmMap[.Name]} + ": \"" + .parameterValue + "\""))
  | join("\n")
' < example.json;

I want/expected the output to be:

ssm_theGreatKEYAlias: "magicvaluenotrelevant"
ssm_apiKey: "singleblabla"

With the process.sh script that I provided I get error because cannot find a way to use the associative array sshMap inside jq map, to transform the parameter name from json into the mapped alias from sshMap.

Any idea how that can be achieved ?

If I change the line 13 of process.sh into

  | map((.Name + ": \"" + .parameterValue + "\""))

it works fine but without mapping, uses original parameter name as comes from json file.

ikegami · Accepted Answer · 2022-04-25T16:53:09.757

1

You're trying to access a shell var from jq.

Fixed:

ssm_map='
   {
      "some_great_key": "theGreatKEYAlias",
      "api_magic":      "apiKey"
   }
'

jq -r --argjson ssm_map "$ssm_map" '
   .Parameters[] |
   "ssm_\( $ssm_map[ .Name ] ): \"\( .Value )\""
'

Demo on jqplay

You could convert the variable you describe into JSON with varying levels of difficulty (depending on what limitations, if any, you're willing to accept). But since you're building the variable from the output of yq, should could simply have it output JSON directly.

edited Apr 25 '22 at 16:53

answered Apr 25 '22 at 16:13

ikegami

367,544
15
269
518

thank you, the real situation is a little more complex. I am reading the ssmMap from a yaml file and is only an associative array, so I don't have it into one json format like you declare there. Is there some way to transform ssmMap from associative bash array into the string – Kristi Jorgji Apr 25 '22 at 16:16
Re "*I am reading the ssmMap from a yaml file*", If you have problems building JSON associative array instead of a bash associative array, feel free to research and post a question about that. – ikegami Apr 25 '22 at 16:19
I am not having problems with anything. I just simplified this question instead of providing the details of how I form the ssmMap, I provided hardcoded example. So the answer should start by assuming that I cannot change that fact, but I can use `ssmMap` as associative array , of course including possible transformations – Kristi Jorgji Apr 25 '22 at 16:21
Re "*I am not having problems with anything*", You're literally asking for help.... /// Re "*I just simplified this question instead of providing the details of how I form the ssmMap*", But that's the part you need to change. You literally removed everything that's relevant to solving your actual problem. There's nothing for us to work with. – ikegami Apr 25 '22 at 16:22
I mean I don't have problems with reading the ssmMap definition. And of course I have problems and I need help with rest therefore the question. I am researching now how to map the associative array `ssmMap` into a json string like the one provided in your answer, then will execute the modified script with the answer – Kristi Jorgji Apr 25 '22 at 16:24
I haven't tried something yet, I just started to look how to form json string from bash array. But I am open to your suggestion to read from the yaml file directly into json string if that is easier. So I don't read into ssmMap as assoc array from start but to json string – Kristi Jorgji Apr 25 '22 at 16:26
Bash is not well equipped to generate JSON. `jq` could be used, but I'd recommend skipping that step and generating the json straight from the yaml instead of building a bash array in the middle. – ikegami Apr 25 '22 at 16:27
I filled the ssmMap associative array from the previous question https://stackoverflow.com/questions/72001044/how-to-read-yaml-file-into-bash-associative-array. Now I will look how to read that `map.yaml` file as ssmMap string json that you commented – Kristi Jorgji Apr 25 '22 at 16:27
I think `yq` can emit json? – ikegami Apr 25 '22 at 16:28
Either one is fine: [mikefarah/yq](https://github.com/mikefarah/yq) is a YAML processor and can output JSON using the `@json` operator. [kislyuk/yq](https://github.com/kislyuk/yq) is a YAML wrapper for `jq`, so it automatically transforms the YAML input into JSON and lets you perform any `jq` filter on it. [itchyny/gojq](https://github.com/itchyny/gojq) is a `jq` rewrite in Go, implementing a `--yaml-input` option, which lets you read in YAML instead of JSON. – pmf Apr 25 '22 at 16:46
I converted in some hacky way the yaml file into json string and your solution worked. Marking as answer thank you. I converted this way the yaml to json string https://pastebin.com/CgveFmPL , basically iterated over every file into a folder of yaml definitions. Then parsed value input and created the ssmMap as json string – Kristi Jorgji Apr 25 '22 at 17:00

score 0 · Answer 2 · answered Apr 25 '22 at 19:14

Using mikefarah/yq or kislyuk/yq or itchyny/gojq is definitely the better approach if your actual task is to convert from YAML and to process as JSON.

However, to accomplish the task you have described in the original question, one could add the keys and values of the bash associative array using the --args option and access them within jq via the $ARGS.positional array (requires jq version 1.6), then restore the association by transposeing both halves of that array, and create an INDEX using the key from the first half, which can then be used with JOIN to create the desired text lines, which are output as raw text using the -r option.

jq -r '
  JOIN(
    $ARGS.positional | [.[:length/2], .[length/2:]] | transpose | INDEX(.[0]);
    .Parameters[];
    .Name;
    "ssm_\(.[1][1] // .[0].Name): \(.[0].Value)"
  )
' example.json --args "${!ssmMap[@]}" "${ssmMap[@]}"

ssm_theGreatKEYAlias: magicvaluenotrelevant
ssm_apiKey: blabla

The fallback // .[0].Name is unnecessary for the sample file, but was added to use the original key in case there's an input key not covered by the bash array.

How to use(read) value from associative array during jq map?

2 Answers2