How to convert a subset of YAML into an indexed array of associative arrays?

Question

I am using Bash 4.3 on linux.

I have this simple YAML-esque data file:

products:
  product1:
    name: "Product one"
    price: 100
  product2:
    name: "Product two"
    price: 200
myList:
  - one
  - two

And I need a shell function that, taking the above YAML file as input, can generate and then execute the below Bash code:

unset products product1  product2

# declare the associative arrays
declare -A product1
declare -A product2


# define the data
product1=(
  [name]="Product 1"
  [price]=100
)

product2=(
  [name]="Product 2"
  [price]=200
)

myList=(one two)

# declare the arrays which will contain the names of our associative arrays
products=(product1 product2)

Once I have this wonderful function, I will use the YAML files to automatically generate data, to be used in my custom CMS templating system like so:

{{#foreach product in products}}
  <h3>{{product.name | uppercase}}</h3>
  * {{product.price | money_with_currency £ GBP | without_trailing_zeros}}
{{/foreach}}

I have already tried various YAML parsers, but have not found one that can generate the associative arrays that I need, and some simply didn't work at all (for me, at least):

Most of these, as far as I understand their usage generate things like product_product1_name="foo" :(

Why are you using `bash`, instead of a language with proper data structure support? — chepner, Aug 10 '19 at 15:43
^ can no one else bother asking this please... Obviously if I wanted to/could use Python, Ruby etc, I would... — sc0ttj, Aug 10 '19 at 15:54
@chepner From my OP: > "Once I have this wonderful function, I will use the YAML files to automatically generate data, to be used in my custom CMS templating system like so:" — sc0ttj, Aug 10 '19 at 15:57
The reason the above was asked is that bash **doesn't** have support for nesting data structures. You *cannot* store a bash array (associative or otherwise) inside another array; the closest you can come is storing the *name* of the inner array inside an outer array. Which is to say -- there's hackery that's possible, but it's going to be awful, and slow, and a generally bad idea (complete with lots of namespace pollution). — Charles Duffy, Aug 10 '19 at 16:14
BTW, if you're thinking you're getting this feedback from people who just don't like bash or don't take it seriously, have a look at where chepner and I are on https://stackoverflow.com/tags/bash/topusers — Charles Duffy, Aug 10 '19 at 16:16
...anyhow, is what you're asking for possible? Sure, it's possible. Is implementing it in native bash reasonable? Hell, no. Personally, I'd implement it in `yq`, using the `@sh` jq function to generate correctly-escaped output. — Charles Duffy, Aug 10 '19 at 16:19
@CharlesDuffy I agree with you on the bash applicability. But if the OP can install `yq` which is in Python (which the OP can't/doesn't want to use), it would be easier to make a small python script which output directly generates the bash statements the OP want directly (and source those). — Anthon, Aug 10 '19 at 16:26

Charles Duffy · Accepted Answer · 2019-08-10T17:34:24.337

yaml.sh, which you linked in the question, is a surprisingly good parser. It's a lot easier to convert its output into the format you need than to do anything else.

cleanupValue() {
  local result
  case $1 in
    '"'*'"') result=${1#'"'}; result=${result%'"'} ;;
    "["*"]") result=${1#'['}; result=${result%']'} ;;
    *)       result=$1
  esac
  printf '%s\n' "$result"
}

record_data() {
  local key value="$(cleanupValue "$1")"; shift
  while (( $# >= 2 )); do
    key=$(cleanupValue "$2")
    if (( $# > 2 )); then
      declare -g -a "$1" || continue
      declare -g -A "_${1}__seen" || continue
      local -n __array="$1"
      local -n __seen_array="_${1}__seen"
      if ! [[ ${__seen_array[$key]} ]]; then
        __seen_array[$key]=1
        __array+=( "$key" )
      fi
      unset -n __seen_array
    else
      declare -g -A "$1"    || continue  # "continue" to skip invalid variable names
      local -n __array="$1" || continue
      __array[$key]=$value
    fi
    unset -n __array
    shift
  done
}

while IFS='=' read -r key value; do
  IFS=. read -r -a key_pieces <<<"$key"
  record_data "$value" "${key_pieces[@]}"
done < <(ysh -f your.yml)

# demonstrate results
declare -p products product1 product2 myList

...emits as output:

declare -a products=([0]="product1" [1]="product2")
declare -A product1=([price]="100" [name]="Product one" )
declare -A product2=([price]="200" [name]="Product two" )
declare -A myList=([1]="two" [0]="one" )

That still won't work with my templating system... The `product` variable MUST be an indexed array, containing only the *variable names* of the associative arrays - in this case, 'product1' and 'product2' — sc0ttj, Aug 10 '19 at 17:22
See edit. That said, consider me still very much on the record that this is a bad idea altogether. — Charles Duffy, Aug 10 '19 at 17:34

How to convert a subset of YAML into an indexed array of associative arrays?

1 Answers1

Linked