Adjust netplan yaml with sed or awk

Question

Here is our yaml:

network:
  ethernets:
    ens160:
      addresses:
      - 10.200.2.11/22
      gateway4: 10.200.0.1
      nameservers:
        addresses:
        - 8.8.8.8
        - 4.4.4.4
        search:
        - cybertax.live
  version: 2

I want to change the dns severs only. From:

        - 8.8.8.8
        - 4.4.4.4

to:

        - 10.10.10.10
        - 10.10.10.11

How can I do this? Note: we cannot use or install yq so this needs to be done through sed or awk. Also, yes I know, this is not recommended, but its what needs to be done right now.

What I have tried so far:

sed -i '/        addresses:/,/        search:/ s/^/# /' $netplan_yaml
sed -i '/      nameservers:/a\ \ \ \ \ \ \ \ addresses:' $netplan_yaml
for i in ${!asar_dns[@]}; do
    sed -i "/        addresses:/a\ \ \ \ \ \ \ \ - ${asar_dns[$i]}" $netplan_yaml
done

But this does three things wrong (that I can see).

It matches between addresses and search including the line wiht addresses and search. I only want what is AFTER addresses, and BEFORE search.
It puts the DNS addresses in the associative array between the older addresses that is commented out anywhere there is an "addresses". I dont want to do that on the commented out line.
i dont like how I have to use \ \ \ \ \ \ would much rather use a .* if possible but also need to use the addresses in the associative array.

please update the question to show your attempts at using `sed` and `awk`, along with the (wrong) output generated by your code — markp-fuso, Jan 24 '23 at 16:02
will the ip addresses always be the same? ie, you're alwayts looking to replace `8.8.8.8` and `4.4.4.4`? replacements will always be `10.10.10.10` and `10.10.10.11`? or could any of these ip addresses vary over time? — markp-fuso, Jan 24 '23 at 16:03
could the source ip addresses (`8.8.8.8` and `4.4.4.4` in this case) show up elsewhere in the yaml where they should not be replaced? — markp-fuso, Jan 24 '23 at 16:05
IP addresses maybe anything, and any amount of dns server, cant rely on matching on `8.8.8.8` and/or `4.4.4.4` — Dave, Jan 24 '23 at 16:08
Use a YAML tool like `yq`; as you clearly already discovered, regex solutions tend to be extremely brittle. — tripleee, Jan 25 '23 at 06:18

Jetchisel · Answer 1 · 2023-03-28T01:24:37.250

First make sure that both the script and the file in question does not have windows line endings see are-shell-scripts-sensitive-to-encoding-and-line-endings

Now if ed is available/acceptable, with your given input.

Something like.

ed -s file.yaml <<-'EOF'
  H
  g/nameservers:/;/addresses:/;/search:/-s/[[:digit:].]\{1,\}/10.10.10.11/\
  -s/[[:digit:].]\{1,\}/10.10.10.10/
  ,p
  Q
EOF

Using variables for the values, something like:

#!/bin/sh

dns1=10.10.10.10
dns2=10.10.10.11

ed -s file.yaml <<-EOF
  H
  g/nameservers:/;/addresses:/;/search:/-s/[[:digit:].]\{1,\}/$dns2/\\
  -s/[[:digit:].]\{1,\}/$dns1/
  ,p
  Q
EOF

The file.yaml with 4 ip addresses.

network:
  ethernets:
    ens160:
      addresses:
      - 10.200.2.11/22
      gateway4: 10.200.0.1
      nameservers:
        addresses:
        - 8.8.8.8
        - 4.4.4.4
        - 1.1.1.1
        - 3.3.3.3
        search:
        - cybertax.live
  version: 2

To delete the lines with ip addresses, something like:

#!/bin/sh

ed -s file.yaml <<-EOF
  H
  g/nameservers:/;/addresses:/;/[[:digit:].]\{1,\}/;/search:/-d
  ,p
  Q
EOF

Output

network:
  ethernets:
    ens160:
      addresses:
      - 10.200.2.11/22
      gateway4: 10.200.0.1
      nameservers:
        addresses:
        search:
        - cybertax.live
  version: 2

To delete all ip addresses and add/change two entries, something like:

#!/bin/sh

dns1=10.10.10.10
dns2=10.10.10.11

ed -s file.yaml <<-EOF
  H
  g/nameservers:/;/addresses:/;/[[:digit:].]\{1,\}/;/search:/-d
  t.
  -1s/^\([[:blank:]]\{1,\}\).*/\1- $dns1/
  t.
  .s/^\([[:blank:]]\{1,\}\).*/\1- $dns2/
  ,p
  Q
EOF

Output

network:
  ethernets:
    ens160:
      addresses:
      - 10.200.2.11/22
      gateway4: 10.200.0.1
      nameservers:
        addresses:
        - 10.10.10.10
        - 10.10.10.11
        search:
        - cybertax.live
  version: 2

Change Q to w if in-place editing is required, ala sed -i
Remove the ,p to silence the output.

This work pretty well. Only one problem, which was not a main requirement of mine in the original post, what if the yaml has more than 2 addresses listed? Lets say there are 4 addresses. Running your script replaces the first two entries with "10.10.10.11" the third with "10.10.10.10" and the 4th with "10.10.10.11". Is there a way to modify your solution to wipe out all entries and only put the two dns servers listed? — Dave, Mar 25 '23 at 19:09
Thanks for the edits, but now I am confused. when trying to run the commands all I get it `?` from ed. Even the early commands you shared. Do you know why that would be happening? how can even trouble shoot this `?` is not very helpful... — Dave, Mar 27 '23 at 14:14

score 0 · Answer 2 · answered Jan 24 '23 at 16:38

I found a very hacky solution but it works. Open for feedback.

netplan_yaml=/etc/netplan/00-installer-config.yaml
baddns="$(sed -n '/.*nameservers:/,/.*search:/p' /etc/netplan/00-installer-config.yaml | grep -v 'nameservers\|addresses' | grep -v 'search' | grep -v 10.200 | awk '{print$2}')"
mapfile -t arr_baddns  <<<$baddns
for i in ${arr_baddns[@]}; do
    sed -i "/        - $i/s/^/#/g" $netplan_yaml
done
for i in ${!asar_dns[@]}; do
    sed -i "/        addresses:/a\ \ \ \ \ \ \ \ - ${asar_dns[$i]}" $netplan_yaml
done

Adjust netplan yaml with sed or awk

What I have tried so far:

2 Answers2