How can I match a Russian word using preg_replace in PHP?

Question

How do I go about matching a Russian word in a string (also in Russian) in PHP?

So for example something like this:

$pattern = '/слово/';
preg_replace($pattern, $replacement, $string_in_russian)

I tried utf8_encode and htmlentities with UTF-8 flag for $pattern, but it didn't work. Should I also encode $string_in_russian?

Update: Suggestion for /u flag didn't work so I'm putting the actual code I need this for. It is from a glossary plugin for Wordpress (my site is properly setup to use Russian language, and it does work, but not in this instance). So here's the code

$glossary_title = $glossary_item->post_title;
$glossary_search = '/\b'.$glossary_title.'s*?\b(?=([^"]\*"[^"]\*")\*[^"]*$)/iu';
$glossary_replace = '&lt;a'.$timestamp.'&gt;$0&lt;/a'.$timestamp.'&gt;';
$content_temp = preg_replace($glossary_search, $glossary_replace, $content, 1);

When I do a quick echo into HTML comment this is the kind of string I get for the pattern
/\bсловоs*?\b(?=([^"]*"[^"]")[^"]*$)/iu

And well, that still doesn't seem to work. I thought maybe it was the "s" that was screwing me over (this level of regex is a bit beyond me but I assume it's there for possible plurals), but removing it didn't help.

Update #2: Okay so I decided to do a complete "blank slate" test - plain PHP file with some $content strings in English and Russian and target words to replace. Here is the code

$content_en = 'Nulla volutpat pretium nunc, ac feugiat neque lobortis vitae. In eu sapien sit amet eros tincidunt viverra. <b style="color:purple">Proin</b> congue hendrerit felis, et consequat neque ultrices lobortis. <b style="color:purple">Proin</b> luctus bibendum libero et molestie. Sed tristique lacus a urna semper eget feugiat lacus varius. Donec vel sodales diam. <b style="color:purple">Proin</b> fringilla laoreet purus, a facilisis nisi porttitor vel. Nullam ac justo ac elit laoreet ullamcorper vel a magna. Suspendisse in arcu sapien.';
$find_en = 'proin';
$replace_with_en = '<em style="color:red">REPLACEMENT</em>';
$glossary_search = '/\b'.$find_en.'s*?\b(?=([^"]*"[^"]*")*[^"]*$)/iu';
$content_en_replaced = preg_replace($glossary_search, $replace_with_en, $content_en);

$content_ru = 'Lorem Ipsum используют потому, что тот обеспечивает более или менее стандартное заполнение шаблона, а также реальное распределение букв и пробелов в абзацах, которое не получается при простой дубликации "Здесь <b style="color:purple">ваш</b> текст.. Здесь <b style="color:purple">ваш</b> текст.. Здесь <b style="color:purple">ваш</b> текст.." Многие программы электронной вёрстки и редакторы HTML используют Lorem Ipsum в качестве текста по умолчанию.';
$find_ru = 'ваш';
$replace_with_ru = '<em style="color:red">Многие</em>';
$glossary_search = '/\b'.$find_ru.'s*?\b(?=([^"]*"[^"]*")*[^"]*$)/iu';
$content_ru_replaced = preg_replace($glossary_search, $replace_with_ru, $content_ru);

And here is a screenshot of the output http://www.flickr.com/photos/iliadraznin/5372578707/

As you can see the English text had the target word replaced, while the Russian hasn't and the code is identical and I'm using the /u flag. The file is also UTF-8 encoded. Any suggestions? (and again, I tried removing the "s", still nothing)

In soviet russia, russian word regex match YOU! – goat Jan 20 '11 at 18:02 — goat, Jan 20 '11 at 18:02

score 7 · Accepted Answer · edited May 23 '17 at 12:16

7

If you do a real blank slate test, you will find there's nothing wrong with the Russian - it's actually the word boundary aspect that is breaking the regex.

$glossary_search = '/'.$find_ru.'/iu'; // Works fine
$glossary_search = '/\b'.$find_ru.'\b/iu'; // Breaks

Word boundary shorthand is not UTF-8 aware, so, per this question: php regex word boundary matching in utf-8 you can try the following:

$glossary_search = '/(?<!\pL)'.$find_ru.'(?!\pL)/iu';

That works fine on my test here.

edited May 23 '17 at 12:16

Community

1
1

answered Jan 20 '11 at 17:34

tomwalsham

781
4
4

Ahhh I see. Never would've thought it had anything to do with word boundary. Thanks. It still seems that the rest of the expression - (?=([^"]*"[^"]*")*[^"]*$) - breaks it, but I can use different expressions based on language so that shouldn't be a problem. – Ilia Draznin Jan 20 '11 at 17:41
A word‐boundary via `\b` is actually the same as `(?:(?<=\w)(?!\w)|(?<!\w)(?=\w))`, while a non–word‐boundary `\B` is the same as `(?:(?<=\w)(?=\w)|(?<!\w)(?!\w))`. If you only want `\w` to match the Unicode Alphabetic property, you have to use `[\pL\pM\p{Nl}]` instead; otherwise it also matches decimal numbers and connector punctuation. – tchrist Jan 20 '11 at 18:34
1

To make `\w` work correctly in Java requires `[\pL\pM\p{Nd}\p{Nl}\p{Pc}[\p{InEnclosedAlphanumerics}&&\p{So}]]`, but you can’t do conjunctive charclass in php. But you can probably live without the trailing bit, so just `[\pL\pM\p{Nd}\p{Nl}\p{Pc}]`. That means `\b` would have to be `(?:(?<=[\pL\pM\p{Nd}\p{Nl}\p{Pc}])(?![\pL\pM\p{Nd}\p{Nl}\p{Pc}])|(?<![\pL\pM\p{Nd}\p{Nl}\p{Pc}])(?=[\pL\pM\p{Nd}\p{Nl}\p{Pc}])`. Yes, that’s ridiculous but you have no choice when your language is so b0rked up that it lacks proper Unicode support. **LOBBY TO FIX THIS NEGLIGENT OVERSIGHT!** – tchrist Jan 20 '11 at 18:41

score 1 · Answer 2 · answered Jan 20 '11 at 03:36

1

For starters, you must make sure that your php file is encoded with UTF-8. Even if you do not have any UTF-8 characters in the file (they might be passed in from another file), the file must be UTF-8 for functions inside of it to work with UTF-8.

answered Jan 20 '11 at 03:36

Joel

2,654
6
31
46

I checked, it is UTF-8, at least when I save it in Notepad++. And the entire site works just fine in Russian - URLs, permalinks, I can search, heck, even the YARP plugin seems to work. It's gotta be just a syntax thing in the regex, something is missing or is incorrect. :-/ – Ilia Draznin Jan 20 '11 at 13:12
Could you please post a string that passes the expression? In ASCII, for instance. Just to rule the encoding out. – Dennis Kreminsky Jan 20 '11 at 13:38
By "passes the expression" you mean works? Any English word works, at least the few I tried - if it's in $content it finds it and replaces it with $glossary_replace. I only have trouble with Russian words. So "/\bwords*?\b(?=([^"]*"[^"]")[^"]*$)/i" works – Ilia Draznin Jan 20 '11 at 16:22
What are you trying to capture with that regex? I'm just curious because I ran your script simplified, so that it's just `$glossary_search = '/'.$find_ru.'/iu';` and it works. – Ben Saufley Jan 20 '11 at 17:17
I'll be honest, I don't know :) This is a script taken from a glossary plugin for wordpress. I'm not expert on regex so I assume this is needed, maybe to account for potential html tags, or formatting tags, or something like that. – Ilia Draznin Jan 20 '11 at 17:19

score 0 · Answer 3 · answered Feb 25 '11 at 14:53

"u" option in PCRE regexp's provides Unicode, so:

<?php
  $str = 'тест бла бла бла';
  if(preg_match("'тест'isu", $str, $match))
  {
    echo $match;
  }
?>

Also, example for preg_replace:

<?php
  $str = 'тест бла бла бла';
  echo preg_replace("'бла'isu", '', $str);
?>

How can I match a Russian word using preg_replace in PHP?

3 Answers3

Linked