How to flatten the YAML structure (List/Map)

Question

Is it possible to convert hierarchical attributes(Map/List) to flattened as linear YAML structure?

For instance, there is a below property defined in a microservice:

default:  
  app:
    authentication: 
      security: ${app.authentication.security}

The above can be overridden using kube object i.e. configMap and as the above property exposed as ${app.authentication.security}, so inside configMap we aren't required to define the entire structure(shown as below), so here we have flattered the YAML hierarchy. We are not required to define the entire order.

app.authentication.security: false

Now, is there any possibility to flatter the YAML Map/List, and those Map/List properties can be dynamic. Below is the sample structure.

Example:

app-ex:
  list: 
    - key1: value
      key2: value
      key3: 
        - list1
        - list2
        - list3
    - key4: value
      key5: value
      key6: 
        - list1
        - list2
        - list3
    .
    .
    - key N

The possible expected output can be a flat pattern as below:

app-ex.list.key1: val1
app-ex.list.key2: val2

[yq can do this](https://stackoverflow.com/a/69303779/347964). I don't know why you just drop mid-level keys (such as `app.security` instead of `app.authentication.security`, or the list indexes inside of `list`). If there is some logic behind that, you'll probably need to implement it in a real programming language. — flyx, Sep 13 '22 at 14:05
The first two YAML files you show are very different and do not identify the same data structure. You could not use them interchangeably in either Kubernetes YAML structures or Helm chart values. Are you using an application framework (Spring?) that allows either format? — David Maze, Sep 15 '22 at 13:24

score 0 · Answer 1 · answered Sep 25 '22 at 19:21

YAML processors can handle and manipulate such files with ease. kislyuk/yq could be one choice. Here's a solution with it using tostream to generate path-value pairs, and from_entries to generate the output object:

yq -y '
  [tostream | select(has(1)) | first |= join(".") | {key: first, value: last}]
  | from_entries
' file.yaml

app-ex.list.0.key1: value
app-ex.list.0.key2: value
app-ex.list.0.key3.0: list1
app-ex.list.0.key3.1: list2
app-ex.list.0.key3.2: list3
app-ex.list.1.key4: value
app-ex.list.1.key5: value
app-ex.list.1.key6.0: list1
app-ex.list.1.key6.1: list2
app-ex.list.1.key6.2: list3

How to flatten the YAML structure (List/Map)

1 Answers1