Replace version inside YAML file

Question

I have this YAML file:

openapi: 3.0.0
info:
  title: Test
  description: >-
    This is the official REST API documentation for all available routes
    provided by the **Test** plugin.
  version: 2.0.0
servers:
  - url: https://entwicklungsumgebung
components:
  securitySchemes:
    basicAuth:
      type: http

Inside my CI pipeline, I'm trying to modify the version at line 7. At the middle of the file, there are multiple keys named test_version or prod_version which should not be replaced. For this, I've written this sed command:

sed -i '' 's/^version: .*/version: 3.0.0/' test.yml

Initially, I've not used ^ after s/ but this matched also everything before version. Now, nothing gets replaced. Any idea what I'm doing wrong?

score 2 · Accepted Answer · answered Feb 15 '23 at 17:55

2

You have to preserve the whitespace, so you either have to match the entire line and use a capture group:

sed -i '' 's/^\([[:blank:]]*version:\) .*/\1 3.0.0/' test.yml

Or you could match only version, with the risk of matching where it shouldn't:

sed -i '' 's/\(version:\) .*/\1 3.0.0/' test.yml

A more robust solution would be to use a tool what can parse YAML, such as yq:

yq -i '.info.version = "3.0.0"' test.yml

answered Feb 15 '23 at 17:55

Benjamin W.

46,058
19
106
116

Solution 1 seems to match also my other versions like: `new_version`, `installed_version` – Mr. Jo Feb 15 '23 at 18:47
@Mr.Jo I don't see how that's possible, the regular expression is anchored at the line start and allows only blanks before `version`. Did you skip the `^`? – Benjamin W. Feb 15 '23 at 20:34
No, I executed it 1:1. Maybe it's because I'm testing on a MacBook? – Mr. Jo Feb 15 '23 at 20:49
If I execute it on my Ubuntu 18.4 (I know, EOL comes) I'm getting another error: `sed: can't read s/^\([[:blank:]]*version:\) .*/\1 20.0.0/: No such file or directory` The file is directly inside the execution path of the sed command! – Mr. Jo Feb 15 '23 at 20:54
If I remove the `''` after the`sed` command on Ubuntu, everything works! Also, the correct version gets changed. Thanks a lot! – Mr. Jo Feb 15 '23 at 20:58
1

@Mr.Jo Oh, you're on Linux, then yes, the `''` are not required there, but they are required for macOS sed. This comes down to basically [this question](https://stackoverflow.com/q/5694228/3266847). – Benjamin W. Feb 15 '23 at 21:39
1

And [this answer](https://stackoverflow.com/a/24276470/3266847) is an excellent overview of all the differences. – Benjamin W. Feb 15 '23 at 21:41

Zak · Answer 2 · 2023-02-15T17:57:46.790

1

I would use a find .. To find all your YML files .. Then, there is no need for the ^ in your regex. I also use the -i flag for suffixing as a precaution, and /g for "global" just in case version: is mentioned more than one time in the file --

edit Also leave the white space before version -- in the replacement to preserve the indention

find ./ -name '*.yml' -exec sed -i "s/ version: .*/ version: 3.0.0/g" {} \;

So this file BEFORE:

openapi: 3.0.0
info:
  title: Test
  description: >-
    This is the official REST API documentation for all available routes
    provided by the **Test** plugin.
  test_version: 5.2.2
  prod_version: 4.3.3
 version: 2.0.0
servers:
  - url: https://entwicklungsumgebung
components:
  securitySchemes:
    basicAuth:
      type: http

OUTPUT:

openapi: 3.0.0
info:
  title: Test
  description: >-
    This is the official REST API documentation for all available routes
    provided by the **Test** plugin.
  test_version: 5.2.2
  prod_version: 4.3.3
 version: 3.0.0
servers:
  - url: https://entwicklungsumgebung
components:
  securitySchemes:
    basicAuth:
      type: http

edited Feb 15 '23 at 17:57

answered Feb 15 '23 at 17:40

Zak

6,976
2
26
48

I'm getting an error: `find ./ -name 'test.yml' -exec sed -i "s/ version: .*/ version: 3.0.0/g" {} \; sed: 1: ".//test.yml": invalid command code .` – Mr. Jo Feb 15 '23 at 18:46
Depending on your version of `sed` .. Try it without the `-i` flag .. -- Tested on Ubuntu 20.04 with the exact contents above copied and pasted. – Zak Feb 15 '23 at 18:51
You can also try JUST the `sed` command `sed -i "s/ version: .*/ version: 3.0.0/g" test.yml` -- Also tested and functioning .. – Zak Feb 15 '23 at 18:56
I was testing locally on my Mac. Maybe it’s different there. I’ll test on my Ubuntu target server in 1 1/2 hours and let you know the results! Maybe there is something different at macOS – Mr. Jo Feb 15 '23 at 19:12
1

Mac `sed` is slightly different -- I am not versed .. But I know that there are slight syntactical differences. -- including the `-s` and `-i` flags. My solution will work on your production machine. – Zak Feb 15 '23 at 19:14
I'm unable to test it inside my GitLab CI/CD Pipeline because if I copy it into my YAML CI file, my IDE says the following and GitLab execution fails: `Schema validation: Incompatible types. Required one of: array, string. Actual: object. ` – Mr. Jo Feb 15 '23 at 20:51
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/251907/discussion-between-mr-jo-and-zak). – Mr. Jo Feb 15 '23 at 20:58
The above answer works now for me but I love your approach and would like to test it as well! The find can be useful for different cases. Let's try to get it running as well. – Mr. Jo Feb 15 '23 at 20:59
@Mr.Jo see chat – Zak Feb 15 '23 at 21:12

Replace version inside YAML file

2 Answers2