Multiple matches in a string using regex in bash

Question

Been looking for some more advanced regex info on regex with bash and have not found much information on it.

Here's the concept, with a simple string:

myString="DO-BATCH BATCH-DO"

if [[ $myString =~ ([[:alpha:]]*)-([[:alpha:]]*) ]]; then
 echo ${BASH_REMATCH[1]} #first perens
 echo ${BASH_REMATCH[2]} #second perens
 echo ${BASH_REMATCH[0]} #full match
fi

outputs:
BATCH
DO
DO-BATCH

So fine it does the first match (BATCH-DO) but how do I pull a second match (DO-BATCH)? I'm just drawing a blank here and can not find much info on bash regex.

Answer 1

OK so one way I did this is to put it in a for loop:

myString="DO-BATCH BATCH-DO"
for aString in ${myString[@]}; do
    if [[ ${aString} =~ ([[:alpha:]]*)-([[:alpha:]]*) ]]; then
     echo ${BASH_REMATCH[1]} #first perens
     echo ${BASH_REMATCH[2]} #second perens
     echo ${BASH_REMATCH[0]} #full match
    fi
done

which outputs:
DO
BATCH
DO-BATCH
BATCH
DO
BATCH-DO

Which works but I kind of was hoping to pull it all from one regex if possible.

Answer 2

In your answer, myString is not an array, but you use an array reference to access it. This works in Bash because the 0th element of an array can be referred to by just the variable name and vice versa. What that means is that you could use:

for aString in $myString; do

to get the same result in this case.

In your question, you say the output includes "BATCH-DO". I get "DO-BATCH" so I presume this was a typo.

The only way to get the extra strings without using a for loop is to use a longer regex. By the way, I recommend putting Bash regexes in variable. It makes certain types much easier to use (those the contain whitespace or special characters, for example.

pattern='(([[:alpha:]]*)-([[:alpha:]]*)) +(([[:alpha:]]*)-([[:alpha:]]*))'
[[ $myString =~ $pattern ]]
declare -p BASH_REMATCH    #dump the array

Outputs:

declare -ar BASH_REMATCH='([0]="DO-BATCH BATCH-DO" [1]="DO-BATCH" [2]="DO" [3]="BATCH" [4]="BATCH-DO" [5]="BATCH" [6]="DO")'

The extra set of parentheses is needed if you want to capture the individual substrings as well as the hyphenated phrases. If you don't need the individual words, you can eliminate the inner sets of parentheses.

Notice that you don't need to use if if you only need to extract substrings. You only need if to take conditional action based on a match.

Also notice that ${BASH_REMATCH[0]} will be quite different with the longer regex since it contains the whole match.

Answer 3

Per @Dennis Williamson's post I messed around and ended up with the following:

myString="DO-BATCH BATCH-DO" 
pattern='(([[:alpha:]]*)-([[:alpha:]]*)) +(([[:alpha:]]*)-([[:alpha:]]*))'

[[ $myString =~ $pattern ]] && { read -a myREMatch <<< ${BASH_REMATCH[@]}; }

echo "\${myString} -> ${myString}" 
echo "\${#myREMatch[@]} -> ${#myREMatch[@]}"

for (( i = 0; i < ${#myREMatch[@]}; i++ )); do   
  echo "\${myREMatch[$i]} -> ${myREMatch[$i]}" 
done

This works fine except myString must have the 2 values to be there. So I post this because its is kinda interesting and I had fun messing with it. But to get this more generic and address any amount of paired groups (ie DO-BATCH) I'm going to go with a modified version of my original answer:

myString="DO-BATCH BATCH-DO" 
myRE="([[:alpha:]]*)-([[:alpha:]]*)"

read -a myString <<< $myString

for aString in ${myString[@]}; do   
  echo "\${aString} -> ${aString}"  
  if [[ ${aString} =~ ${myRE} ]]; then
    echo "\${BASH_REMATCH[@]} -> ${BASH_REMATCH[@]}"
    echo "\${#BASH_REMATCH[@]} -> ${#BASH_REMATCH[@]}"
    for (( i = 0; i < ${#BASH_REMATCH[@]}; i++ )); do
      echo "\${BASH_REMATCH[$i]} -> ${BASH_REMATCH[$i]}"
    done
  fi
done

I would have liked a perlre like multiple match but this works fine.

Answer 4

Although this is a year old question (without accepted answer), could the regex pattern be simplified to:

myRE="([[:alpha:]]*-[[:alpha:]]*)"

by removing the inner parenthesis to find a smaller (more concise) set of the words DO-BATCH and BATCH-DO?

It works for me in you 18:10 time answer. ${BASH_REMATCH[0]} and ${BASH_REMATCH[1]} result in the 2 words being found.

Answer 5

In case you don't actually know how many matches there will be ahead of time, you can use this:

#!/bin/bash

function handle_value {
  local one=$1
  local two=$2

  echo "i found ${one}-${two}"
}

function match_all {
  local current=$1
  local regex=$2
  local handler=$3

  while [[ ${current} =~ ${regex} ]]; do
    "${handler}" "${BASH_REMATCH[@]:1}"

    # trim off the portion already matched
    current="${current#${BASH_REMATCH[0]}}"
  done
}

match_all \
  "DO-BATCH BATCH-DO" \
  '([[:alpha:]]*)-([[:alpha:]]*)[[:space:]]*' \
  'handle_value'