Perl regex replacing at word boundary. Detecting "/" as a word boundary

Question

I am running into a strange regex issue.... I have a document where I am doing a replace... as an example I want to replace "DEXX" with "DEXX/AREX" and then with the next substitution replace... "AREX" with "AREX/CUBE"

DEXX and AREX are stored in a hash like so.... "DEXX" => "AREX", "AREX" => "CUBE"

The regex I have is this.....

foreach (keys %hashstore){
    $doc=~s!\b($_)\b!$1/$hashstore{$_}!ig;
}

What's happening is that "DEXX" is being replaced with "DEXX/AREX" ok but when "DEXX/AREX" is encountered the regex is replacing "DEXX/AREX" with "DEXX/AREX/CUBE" when it should only be replacing "AREX" when it finds it as a standalone word not as part of another combination like "DEXX/AREX"

It seems to detect "/" as a word boundary. Has anyone encountered this or know of a fix around it? Many thanks! Amy

Answer 1

But / is a word boundary. From perldoc perlreref:

\b Match word boundary (between \w and \W).

In light of your comment below, you should avoid the loop:

#!/usr/bin/perl

use strict; use warnings;
use Regex::PreSuf;

my %lookup = (
    "DEXX" => "AREX",
    "AREX" => "CUBE",
);

my $doc = 'DEXX AREX AREX DEXX AREX DEXX DEXX DEXX AREX';
my $re = presuf keys %lookup;

$doc =~ s{($re)}{$1/$lookup{$1}}g;

print $doc, "\n";

Output:

DEXX/AREX AREX/CUBE AREX/CUBE DEXX/AREX AREX/CUBE DEXX/AREX DEXX/AREX DEXX/AREX
AREX/CUBE

Of course, you don't have to use Regex::PreSuf if you only have two keys:

s{(AREX|DEXX)}{$1/$lookup{$1}}g;

will also do. But, for a longer list of keys, I find Regex::PreSuf to be very handy.

Update: Of course, if the keys can occur in any case in the text, you can use uc to transform when looking up the replacement:

So, either

$doc =~ s{($re)}{join '/', uc($1), $lookup{uc $1}}eig;

or

$doc =~ s{($re)}{join '/', $1, $lookup{uc $1}}eig;

depending on what you need.

Also, ysth points out in the comments "With 5.10 and later, Regex::PreSuf generates a poorer regex than the naive alternation in most cases." So,

my $re = join '|', map quotemeta, sort { length($b) <=> length($a) } keys %lookup; 

might be better. The sort is needed if some keys might be initial substrings of other keys.

Answer 2

The word boundary is any time there's a transition between \w and \W, which is to say [a-zA-Z0-9_] and [^a-zA-Z0-9_] if you're dealing with ASCII.

You should be able to get around this problem by using a negative lookbehind:

foreach (keys %hashstore){
    $doc=~s!(?<!/)\b($_)\b!$1/$hashstore{$_}!ig;
}

Answer 3

\b is equivalent to (though more efficient than) (?:(?<!\w)(?=\w)|(?<=\w)(?!\w)). If you want a different set of word characters than the default, just use that but with \w replaced by an appropriate character class.

Answer 4

First off I am indebted to Sinan (who isn't in regards to Perl on SO? I know I have been lurking a long time....) and ysth. Thanks to these two I have a better grasp of regexes. My solution however was the following...

my $pat = join '|', keys(%hashstore);
$doc =~ s!\b($pat)\b!$1/$hashstore{uc($1)}!ig;

Problem that I had was I replacing my replacements! Normally I really try to hash these things out but this was such a tight deadline and Sinan and ysth, you both rock severely! Amy

Answer 5

Boundaries from the standpoint of \b are often not quite what you want, especially given that English words can contain apostrophes and dashes, and that these behave very differently than letters do when you put a \b next to them. See this answer for more explanation of this issue, and what to do about it.