Regex to replace pairs %s, \s correctly even with %% and \\

Question

I would like to find a regex that would replace %% with % and %s with my custom string foobar. This is tricker that it sounds, because it should turn %%s into %s and not %foobar, so this naive implementation does not work:

s/%%/%/g
s/%s/foobar/g

This problem is quite common and I've come across it multiple times in my programming life. Not just percent s or percent escaping, but also backslash character or backlash escaping. I'm going to post my usual solution but I'm wondering if there's a better way.

(Allow me to do some keyword stuffing for my future searches: character pairs, backslash backslash, backslash x, percent percent, percent s. Thank you.)

If there are specific language features that would help in this use-case, I'd be interested in hearing what they are.

Example input and output:

input  : test %%, test %s, test %%s too
output : test %, test foobar, test %s too

Another one:

input  : test%%,test%s,test%%stoo
output : test%,testfoobar,test%stoo

Answer 1

This can be simplified and can be done in single replace call:

var str = "test %s, test %%, test %%s too";
var output = str.replace(/%%|(%s)/g, function($0, $1){
     return $1!==undefined?'foobar':'%'; });
//=> test foobar, test %, test %s too

We use alternation first /%%/(%s)/ and use a capturing group while matching (%s). In the replace callback we use $1!==undefined to decide what string to be used as replacer.

Answer 2

The thing with regular expressions is - if you run them twice, they get applied twice.

So yes - you're implementation isn't going to work, because you 'search twice' - after your first replace, you have no way to tell the difference.

So how about instead;

#!/usr/bin/env perl

use strict;
use warnings;

my %replace = ( '%%' => '%',  
                '%s' => 'foobar' );

my $search = join ( "|", keys %replace );
   $search = qr/($search)/; 

print "Search regex: $search\n";
while ( <DATA> ) {
   s/$search/$replace{$1}/g;
   print;
}

##output : test %, test foobar, test %s too
##output : test%,testfoobar,test%stoo

__DATA__
test %%, test %s, test %%s too
test%%,test%s,test%%stoo

That's doing it perlishly, but you're building a lookup table - capturing the left hand side, and looking up what it should replace with on the right. (You can turn this into a one liner too).

Output:

Search regex: (?^:(%%|%s))
test %, test foobar, test %s too
test%,testfoobar,test%stoo

Pretty sure you should be able to implement this in most languages.

As an alternative, it's probably worth considering regex lookaround which lets you - if you do your regexs in the opposite order:

#!/usr/bin/env perl

use strict;
use warnings;

while ( <DATA> ) {
   s/(?<!%)%s/foobar/g;
   s/%%/%/g;
   print;
}

##output : test %, test foobar, test %s too
##output : test%,testfoobar,test%stoo

__DATA__
test %%, test %s, test %%s too
test%%,test%s,test%%stoo

(?<!%) is a zero width assertion that says 'not preceeded by a percent' - so it runs through and replaces just %s with "foobar" (but ignores %%s). And then applies the secondary transform, which doesn't catch 'foobar' because it doesn't have a %% in there.

Output:

test %, test foobar, test %s too
test%,testfoobar,test%stoo

Downside of this approach is that not all languages properly support look around. (It's an 'advanced regex' thing, not 'basic')

Answer 3

The general problem of escape sequences is not optimally solved by regular expression substitution.

You have to think of your string as a sequence of tokens evaluated lexically by a state machine.

You start by being in a NORMAL state.
In the normal state, any character that you encounter is copied-as-is to the output, unless it is a %, in which case you enter a state PERCENT.
In that state, you can encounter a %, then you output % and return to NORMAL.
You can also encounter a s, and then you pop the next substitution string, output it, and return to NORMAL.
Finally depending on the behavior you need, any other character encountered in the PERCENT state can yield an error, or be ignored...

Example javascript code:

function parseString(s, vars) {
    var NORMAL = 0, PERCENT = 1;

    var state = NORMAL;
    var varidx = 0;
    var output = '';
    for (var i = 0; i <  s.length; i++) {
        if (state == NORMAL) {
            if (s[i] == '%') {
                state = PERCENT;
            } else {
                output += s[i];
            }
        } else if (state == PERCENT) {
            if (s[i] == '%') {
                output += s[i];
                state = NORMAL;
            } else if (s[i] == 's') {
                output += vars[varidx++];
                state = NORMAL;
            } else {
                throw 'Invalid syntax';
            }
        }
    }
    return output;
}

Example:

parseString("test %%, test %s, test %%s too", ['foo']);
// returns "test %, test foo, test %s too"

While this approach is more code than regexp-based solutions, it is probably faster because regular expressions involve a much greater degree of complexity, and it allows you to handle invalid syntax the way that fits you best.

Answer 4

This is one easy way of doing it. Split the string into an array of chunks, for lack of better term, using the regex /%%|%s|./, so that each chunk is a character or an escaped character, and then check each individual chunk for %s and %% like this, do the unescaping, and join the array again, like this:

Input  : "test %s, test %%, test %%s too"
Array  : ["t", "e", "s", "t", " ", "%s", ",", " ", "t", "e", "s", "t", " ", "%%",
          " ", "t", "e", "s", "t", "%%", "s", " ", "t", "o", "o"]
Output : "test foobar, test %, test %s too"

Here is the same idea in Javascript without using a variable to hold the chunks:

var str = "test %s, test %%, test %%s too";
var output = str.replace(/%%|%s|./g, function(match, capture) { 
  return match.replace("%%", "%").replace("%s", "foobar");
});
console.log("output:", output);

Answer 5

Here's javascript way to replace %% by % and %s by foobar.

string.replace(/%%|%s/g, function (match) {
    // If %% is matched, replace it by %
    // else %s is matched, replace by `foobar`
    return match === '%%' ? '%' : 'foobar';
});

var str = "test %s, test %%, test %%s too";

str = str.replace(/%%|%s/g, function (_) {
    return _ === '%%' ? '%' : 'foobar';
});

console.log(str);
document.body.innerHTML = str;

Using this approach, String#replace is used only once, instead of three times as in other answers.

---

Here's another approach using the variable swap logic using temp variable. Similar to this answer by @torazaburo

str
    .replace(/%%/g, '2percent') // Replace first string by some string that will not possibly be appear/present in the main string
    .replace(/%s/g, 'foobar') // Replace second string
    .replace(/2percent/g, '%'); // Replace temp by the normal string

var str = "test %s, test %%, test %%s too";

str = str
    .replace(/%%/g, '2percent')
    .replace(/%s/g, 'foobar')
    .replace(/2percent/g, '%');

console.log(str);
document.body.innerHTML = str;