Bash regex ungreedy match

Question

I have a regex pattern that is supposed to match at multiple places in a string. I want to get all the match groups into one array and then print every element.

So, I've been trying this:

#!/bin/bash

f=$'\n\tShare1   Disk\n\tShare2  Disk\n\tPrnt1  Printer'
regex=$'\n\t(.+?)\\s+Disk'
if [[ $f =~ $regex ]]
then
    for match in "${BASH_REMATCH[@]}"
    do
        echo "New match: $match"
    done
else
    echo "No matches"
fi

Result:

New match: 
    Share1   Disk
    Share2  Disk
New match: Share1   Disk
    Share2

The expected result would have been

New match: Share1
New match: Share2

I think it doesn't work because my .+? is matching greedy. So I looked up how this could be accomplished with bash regex. But everyone seems to suggest to use grep with perl regex.

But surely there has to be another way. I was thinking maybe something like [^\\s]+.. But the output for that was:

New match: 
    Share1   Disk
New match: Share1

... Any ideas?

One idea would be to use `[^\\s]+?` instead of `.+?` . That will match characters until a whitespace if found. — Rahul, Dec 14 '16 at 12:56
Both produce the same result as `[^\\s]+` which I have already mentioned in my question. I don't think that the `?` is even supported in bash, I mean in this context.. I mean a `?` behind a `+` usually means `match ungreedy`. — Forivin, Dec 14 '16 at 13:00
Based on this [answer](http://stackoverflow.com/a/20241622/2146346) `POSIX regular expression` (which is what is used with `=~` operator) does not have non-greedy quantifiers. — NarūnasK, Dec 14 '16 at 13:00
@ThomasAyoub: Thanks for pointing out. @Forivin: `[^\s]` is same as `\S`. Use `\\` for escaping if needed. — Rahul, Dec 14 '16 at 13:00
@Forivin: You should use the **first captured group** . Something like `$match[1]` (Not good with bash). — Rahul, Dec 14 '16 at 13:02
@Rahul Well, yes I escaped it propery, but as I said didn't produce the desired result. — Forivin, Dec 14 '16 at 13:04
You should split the string with newline first, then iterate the chunks checking each with your regex and grab the value using `${BASH_REMATCH[1]}`. — Wiktor Stribiżew, Dec 14 '16 at 13:11

Eric Renouf · Accepted Answer · 2016-12-14T13:23:46.607

There are a couple of issues here. First, the first element of BASH_REMATCH is the entire string that matched the pattern, not the capture group, so you want to use ${BASH_REMATCH[@]:1} to get those things that were in the capture groups.

However, bash regex doesn't support repeating the matches multiple times in the string, so bash probably isn't the right tool for this job. Since things are on their own lines though, you could try to use that to split things and apply the pattern to each line like:

f=$'\n\tShare1   Disk\n\tShare2  Disk\n\tPrnt1  Printer'
regex=$'\t(\S+?)\\s+Disk'
while IFS=$'\n' read -r line; do
    if [[ $line =~ $regex ]]
    then
        printf 'New match: %s\n' "${BASH_REMATCH[@]:1}"
    else
        echo "No matches"
    fi
done <<<"$f"

My mind is blown that bash can actually do array slicing (yes I went to figure what in the world is `${arr[@]:1}`) — André Chalella, Jun 15 '23 at 14:08

score 6 · Answer 2 · answered Jan 20 '21 at 11:10

As the accepted answer already states, the solution here is not really to use a non-greedy regex, because Bash doesn't support the notation .*? (it was introduced in Perl 5, and is available in languages whose regex implementation derives from that, but Bash is not one of them). But for visitors finding this question in Google, the answer to the actual question in the title is sometimes to simply use a more limited regex than .* to implement the non-greedy matching you are looking for.

For example,

re='(Disk.*)'
if [[ $f =~ $re ]]; then
 ... # ${BASH_REMATCH[0]} contains everything after (the first occurrence of) Disk

This is just a building block; you would have to take it from there with additional regex matches or a loop. See below for a non-regex variation which does by and large this.

If the thing you don't want to match is a specific character, using a negated character class is simple, elegant, convenient, and compatible back to the dark beginnings of Ken Thompson's original regular expression library. In the OP's example, it looks like you want to skip over a newline and a tab, then match any characters which are not literal spaces.

re=$'\n\t([^ ]+)'

But probably in this case a better solution is to actually use parameter expansions in a loop.

f=$'\n\tShare1   Disk\n\tShare2  Disk\n\tPrnt1  Printer'
result=()
f=${f#$'\n\t'}      # trim any newline + tab prefix
while true; do
  case $f in
    *\ Disk*)
        d=${f%% *}           # capture up to just before first space
        result+=("$d")
        f=${f#*$'\n\t'}     # trim up to next newline + tab
        ;;
    *)
        break ;;
  esac
done
echo "${result[@]}"

See also https://stackoverflow.com/questions/18514135/bash-regular-expression-cant-seem-to-match-any-of-s-s-d-d-w-w-etc for a broader discussion of how to work around the lack of some PCRE regex features in Bash (and more generally POSIX-style regular expressions). — tripleee, Jan 20 '22 at 06:23

Alan Carlyle · Answer 3 · 2022-08-29T08:01:15.593

I came across a very similar problem and solved it in the manner below.

#!/bin/bash

# Captures all %{...} patterns and stops greedy matching by not matching 
# the } inside using [^}] yet capturing it once outside. 
# It also matches all remaining characters.
 
regex="^[^}]*(%{[^}]+})(.*)"

URL="http://%{host}/%{path1}/%{path2}"

value=$URL
matches=()

while true 
do
  if [[ $value =~ $regex ]]
  then 
    matches+=( ${BASH_REMATCH[1]} )
    value=${BASH_REMATCH[2]};
    echo "Yes: ${BASH_REMATCH[1]}  ${BASH_REMATCH[2]}";
  else 
    break; 
  fi
done

echo ${matches[@]}

Output of above will be the following with the last line the array of matches:

$ . loop-match.sh
Yes: %{host}  /%{path1}/%{path2}
Yes: %{path1}  /%{path2}
Yes: %{path2}

%{host} %{path1} %{path2}

Bash regex ungreedy match

3 Answers3