Regex not playing well with Russian text

Question

Possible Duplicate:
regexp with russian lang

I have a regular expression that filters out certain links out of a text and attaches a file icon based on the filetype of the link. Like this:

$text = preg_replace('((<a href="[\w\./:]+getfile.php\?id='.$file.'"([a-zA-Z0-9_\- ,\.:;"=]*)>)([a-zA-Z0-9_,\.:;&\-\(\)\<\>\'/ ]+)</a>)','\\1'.fileicon($name).'</a> \\1\\3</a> ('.($pagecount?$pagecount."&nbsp;".($pagecount>1?$pages:$page1).", ":"").readable_filesize($size,1).')',$text);

this worked great until I tried this with some russian text. The input would be something like:

<a href="/site/getfile.php?id=33">Русский</a>

But it won't show the icon before the link and file information after the link, making me suspect the regex doesn't play well with Russian text. What could be the case here?

Russian letters are not in your character class `[a-zA-Z]+`, so that's why they don't match. — mario, Sep 07 '11 at 09:58
@Gordon: Relevance? A 30.5k user should know that he can either post an answer or not post an answer, but should not post one as a comment... — Lightness Races in Orbit, Sep 07 '11 at 10:18
@ajreal: I can't believe you're the first one to notice that he's parsing HTML with regex! >. — Lightness Races in Orbit, Sep 07 '11 at 10:19
@Tomalak A 32k user should also know that he should not answer duplicates and thus mario correctly provided the answer as a comment only and closevoted with the possible duplicate. — Gordon, Sep 07 '11 at 10:26
@Gordon: No, if you should not answer duplicates then you should not answer duplicates. Answering them incorrectly is not not answering them! — Lightness Races in Orbit, Sep 07 '11 at 11:06
@Tomalak well, technically he didnt *answer* the duplicate. He commented. So, all is well. — Gordon, Sep 07 '11 at 11:09

score 2 · Answer 1 · answered Sep 07 '11 at 09:59

2

Your character class only allows [a-zA-Z0-9_,\.:;&\-\(\)\<\>\'/ ]. There are no russion characters in there.

You can fix this by adding the relevant characters to the class. If you only need to support russian, \p{InCyrillic} should do it. If you want all unicode letters, \p{Letter}.

answered Sep 07 '11 at 09:59

carlpett

12,203
5
48
82

2

The unicode character classes (and `\pL` would suffice) only work if the `/u` modifier is added however. – mario Sep 07 '11 at 10:04

score 2 · Accepted Answer · answered Sep 07 '11 at 10:04

2

You shall use u modifier when working with Unicode strings:

preg_replace('/>([^<]+)</u', '', $string);

answered Sep 07 '11 at 10:04

sanmai

29,083
12
64
76

score 1 · Answer 3 · answered Sep 07 '11 at 11:17

1

You can simplify your regexp down to something like

$re = "~
    (<a\s+href=\".+?getfile\.php\?id=$file\".*?>)
    (.+?)
    </a>
~xui";

this should solve the Cyrillic problem automatically.

answered Sep 07 '11 at 11:17

user187291

53,363
19
95
127

score 0 · Answer 4 · answered Sep 07 '11 at 10:09

0

Cyrillic unicode characters are within the range \x0400-\x04FF. Add this range in your character class.

answered Sep 07 '11 at 10:09

Toto

89,455
62
89
125

Regex not playing well with Russian text

4 Answers4