YAML - substituting values for variables?

Question

consider the following yaml

hadoop:  
   storage: '/x/y/z/a/b'
   streaming_jar_path: '/x/c/d/f/r/*.jar'
   commands:  
       mkdir: 'hadoop dfs -mkdir dir'
       copyFromLocal: 'hadoop dfs -copyFromLocal from_path to_path'  
       run: 'hadoop jar $streaming_jar_path -mapper mapper_path -reducer reducer_path -input hdfs_input -output hdfs_output'

I want to substitute the value of streaming_jar_path to $streaming_jar_path, how can I do that?

I know we can merge the hashes using &(anchors) but here i just want to change one value
I am sorry if this is trivial thing, I am very new to YAML

Thank you

Do you want to rename `streaming_jar_path` to `streaming_jar_path_substitute` in YAML or assign the value of a Python variable called `streaming_jar_path_substitute` to `streaming_jar_path`? — NullUserException, Sep 30 '11 at 16:50
assign the value of streaming_jar_path to streaming_jar_path_substitute — daydreamer, Sep 30 '11 at 16:58
Is your question [Is it possible to do string substitution in YAML?](https://stackoverflow.com/q/30407488/674039) — wim, Mar 09 '18 at 23:19
Some solution, answered here: https://stackoverflow.com/questions/2063616/how-to-reference-a-yaml-setting-from-elsewhere-in-the-same-yaml-file/57739338#57739338 — Andry, Aug 31 '19 at 15:55

Akif · Answer 1 · 2018-03-10T03:59:12.847

You can restructure your YAML file and execute with Ansible.

commands.yml:

- hosts: localhost
  vars:
    streaming_jar_path: '/x/c/d/f/r/*.jar'
  tasks:
    - name: mkdir
      shell: "hadoop dfs -mkdir dir"
    - name: copyFromLocal
      shell: "hadoop dfs -copyFromLocal from_path to_path"
    - name: run
      shell: "hadoop jar {{ streaming_jar_path }} -mapper mapper_path -reducer reducer_path -input hdfs_input -output hdfs_output"

Then simply run ansible-playbook to execute shell commands:

ansible-playbook commands.yml

score 0 · Answer 2 · answered Sep 09 '13 at 19:48

This should be a simple process of reading your file, editing the data and writing back to file.

import yaml

infile = 'input.yaml'
outfile = 'output.yaml'

#read raw yaml data from file into dict
with open(infile, 'r') as f:
    data = yaml.load(f.read())

#make changes to dict
data['hadoop']['streaming_jar_path'] = '$streaming_jar_path'

#write dict back to yaml file
with open(outfile, 'w') as f:
    f.write(yaml.dump(data))

YAML - substituting values for variables?

2 Answers2