How to use `yq` to set a property to a multi-line string?

Question

I have a certificate file that looks similar to:

-------BEGIN CERTIFICATE-------
asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
-------END CERTIFICATE--------

I also have a YAML file (an OpenShift Template) that looks like this:

apiVersion: v1
kind: Template
objects:
- apiVersion: v1
  kind: Route
  tls:
    certificate:
    key:
  someOther: stuff

How can I use yq to set the value of the property certificate in the YAML file above, to the contents of the certificate file so that the output looks somewhat like this:

apiVersion: v1
...
    certificate: |
      -------BEGIN CERTIFICATE-------
      asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
      qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
      -------END CERTIFICATE--------

Neither an Internet search nor documentation was of any help. The closest I got was using the following:

yq w /tmp/template.yaml objects[0].tls.certificate "\n$(cat cert.pem)"

...which left me with the following output:

certificate: !!binary |
      fC0KLS0tLS0tLUJFR0lOIENFUlRJRklDQVRFLS0tLS0tLQphc2RvcWlqZXBvcWp3ZTFpND
      lpMTIwOTQxcDJqNG9tc2zDYXNkYWpzZMNxd2Vxd2UKcXdlbHFqd2vDYXNkbGFqc8O2bGtq
      YXNsZGtqYWtsanNkbGtqYXNkYXNkcGlxd2UKLS0tLS0tLUVORCBDRVJUSUZJQ0FURS0tLS
      0tLS0t

...which strangely is the base64 encoded variant of what I wanted to add preceded by !!binary |. Any ideas what's going on and how I can achieve desired output instead?

Are you still looking for a solution to this problem? In the latest `yq` release v3 - This works `yq w /tmp/template.yaml objects[0].tls.certificate -- "$(< cert.pem)"`, can you let me know if this works? — Inian, Jun 11 '20 at 10:50

score 6 · Answer 1 · answered Dec 05 '21 at 01:12

6

Note that in bash, when you use '$(..)' it will trim trailing newlines (which is why the yaml string block starts with |- instead of |.

To get | (and including the trailing new line) you will need to:

IFS= read -rd '' output < <(cat cert.pem)
output=$output yq e '.objects[0].tls.certificate = strenv(output)' myfile.yml

Disclaimer: I wrote yq

answered Dec 05 '21 at 01:12

mike.f

1,586
13
14

2

You should replace `< <(cat cert.pem)` with `< cert.pem` – Fravadona Jan 10 '22 at 14:09
`yq eval '.objects[0].tld.certificate = load_str("cert.pem")' myfile.yaml` works, too. @mike.f, any reason to prefer your variant and/or is there something wrong with mine? – Armen Michaeli Mar 23 '23 at 15:16

score 2 · Accepted Answer · answered Apr 13 '21 at 00:32

I have tested @Inian suggestion with yq3 and it works.

It can also be achieve in yq4 with the following syntax:

template.yml

# template.yml
apiVersion: v1
kind: Template
objects:
  - apiVersion: v1
    kind: Route
    tls:
      certificate:
      key:
    someOther: stuff

cert.pem

-------BEGIN CERTIFICATE-------
asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
-------END CERTIFICATE--------

command

yq eval '.objects[0].tls.certificate = "'"$(< cert.pem)"'"' template.yml

output

apiVersion: v1
kind: Template
objects:
  - apiVersion: v1
    kind: Route
    tls:
      certificate: |-
        -------BEGIN CERTIFICATE-------
        asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
        qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
        -------END CERTIFICATE--------
      key:
    someOther: stuff

Would you know what is the equivalent in yq2? – Rads Apr 13 '21 at 03:12 — Rads, Apr 13 '21 at 03:12

Armen Michaeli · Answer 3 · 2023-03-29T08:38:49.770

You can use the load_str operator to load a string from a file and assign its result to a property:

yq '.objects[0].tls.certificate = load_str("cert.pem")' template.yaml

(above, template.yaml denotes path to your OpenShift template YAML file, if this wasn't obvious)

The above command will produce the following output in your case:

apiVersion: v1
kind: Template
objects:
  - apiVersion: v1
    kind: Route
    tls:
      certificate: |
        -------BEGIN CERTIFICATE-------
        asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
        qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
        -------END CERTIFICATE--------
      key:
    someOther: stuff

score 0 · Answer 4 · answered Sep 28 '22 at 16:30

I might have found a bug @mike.f

#!/usr/bin/env bash

###
# ❯ yq --version
# yq (https://github.com/mikefarah/yq/) version 4.27.5
##

echo "########## Multiline string is set ##########"
cat > cert.pem << EOF
-------BEGIN CERTIFICATE-------
asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
-------END CERTIFICATE--------
EOF

IFS= read -rd '' cert < <(cat cert.pem)
export cert=$cert
yq --null-input '.cert = strenv(cert)'

echo "########## Multiline string is NOT set ##########"
cat > flatcar_ignition.yml << EOF
storage:
  files:
    - path: /opt/k8s_setup.sh
      filesystem: root
      contents:
        inline: ""
      mode: 0744
      user:
        id: 500
      group:
        id: 501
EOF
cat > k8s_setup.sh << EOF
#!/usr/bin/env bash
echo "test1" > /test.txt
echo "test2" >> /test.txt
echo "test3" >> /test.txt
EOF

IFS= read -rd '' k8s_setup < <(cat k8s_setup.sh)
export k8s_setup=$k8s_setup
yq e '.storage.files[0].contents.inline = strenv(k8s_setup)' flatcar_ignition.yml

Results in:

❯ ./test.sh
########## Multiline string is set ##########
cert: |
  -------BEGIN CERTIFICATE-------
  asdoqijepoqjwe1i49i120941p2j4omslasdajsdqweqwe
  qwelqjwkasdlajsölkjasldkjakljsdlkjasdasdpiqwe
  -------END CERTIFICATE--------

########## Multiline string is NOT set ##########
storage:
  files:
    - path: /opt/k8s_setup.sh
      filesystem: root
      contents:
        inline: "#!/usr/bin/env bash\necho \"test1\" > /test.txt\necho \"test2\" >> /test.txt\necho \"test3\" >> /test.txt\n"
      mode: 0744
      user:
        id: 500
      group:
       id: 501

How to use `yq` to set a property to a multi-line string?

4 Answers4