Bash script is not maching regex

Question

I truly don't understand what I'm missing here. I have a very basic script that need to check if text contains only number (more specific a semantic version for an app), then do some stuff.

The text.txt is something like this:

appVersion: 2.0.1

and this is my script

#!/bin/bash
appVersion=`cat test.txt | awk '/^appVersion:/{print $2}'`
echo "appVersion is " $appVersion
Version=3.1.5

if [[ $appVersion =~ [0-9]+(\.[0-9]+){2,3}$ ]] ; then
    echo "is a number" $appVersion 
 else
    echo "not a number" $appVersion 
fi`

Basically I cat from the text what I need, and then trying to pass that info in an if condition.

The problem is the script is basically not taking the variable appVersion every time I run it, it keep saying that there isn't any number in my variable, like is not matching the regex, but if I pass the variable $Version the script works.

By the way the echo "appVersion is " $appVersion is working (it's printing what I run inside appVersion), so the command inside my variable $appVersion is actually retrieving what I need.

I think I'm missing something with awk but I cannot understand what.

Edit:

I'm running this script on my local pc, on wsl2 ubuntu 20, as a test, to let then run it on linux azure devops agent

Where does `test.txt` come from? Is it actually a YAML file, a TOML file, some other kind of structured file? You nearly always want to use a tool designed to parse structured text, rather than trying to cobble together a regex-based solution. — chepner, Jun 12 '23 at 19:03
What about _(?:[0-9]\.?)+_ ... ? This would match, _123, 1.2.3, 12.3, 1.23, 1.2.3.4,_ etc. — Reilas, Jun 12 '23 at 20:02

Cyrus · Accepted Answer · 2023-06-12T21:56:41.997

3

I suggest to replace

appVersion=`cat test.txt | awk '/^appVersion:/{print $2}'`

with

appVersion=$(awk '/^appVersion:/{print $2}' test.txt | dos2unix)

or with GNU awk

appVersion=$(awk 'BEGIN{RS="\r\n"} /^appVersion:/{print $2}' test.txt)

to convert Windows' carriage-returns.

See: 8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR

edited Jun 12 '23 at 21:56

answered Jun 12 '23 at 18:21

Cyrus

84,225
14
89
153

I'm running this script in local on my wsl2 and I don't have dos2unix installed. – Car_mine Jun 12 '23 at 18:27
whit this one appVersion=$(awk 'BEGIN{RS="\r\n"} /^appVersion:/{print $2}' test.txt) is actually working but I dont get why – Car_mine Jun 12 '23 at 18:34
2

@Car_mine Your text file has DOS/Windows line endings, which cause all sorts of problems with unix tools. In this case, you're getting a mostly-invisible carriage return character at the end of `$appVersion`, which is not a valid part of a version number, so it causes a match failure. See ["Why does my tool output overwrite itself and how do I fix it?"](https://stackoverflow.com/questions/45772525) and ["Are shell scripts sensitive to encoding and line endings?"](https://stackoverflow.com/questions/39527571) – Gordon Davisson Jun 12 '23 at 18:40
... so install it? – Shawn Jun 12 '23 at 18:59
2

multi-char RS as in `RS="\r\n"` is a non-standard extension so if you're writing a script that you'll test on one machine before running it on another machine and expecting it to work then YMMV. You should stick to POSIX standard constructs for that, e.g. `awk '/^appVersion:/{sub(/\r$/,""); print $2}'`, see the URLs @GordonDavisson already provided for more info. – Ed Morton Jun 12 '23 at 19:08
1

@EdMorton: Thank you for pointing this out. I added `with GNU awk` to my question to clarify the safe area of application. – Cyrus Jun 12 '23 at 20:05
Thanks, guys to point this out, I was looping around the POSIX and carriage return but I was not able to understand it properly. – Car_mine Jun 13 '23 at 10:49

Bash script is not maching regex

1 Answers1