Regex to remove non alphanumeric characters from UTF8 strings

Question

How can I remove characters, like punctuation, commas, dashes etc from a string, in a multibyte safe manner?

I will be working with input from many different languages and I am wondering if there is something that can help me with this

Thanks

You want to keep only `a-z` and `0-9`? Or are all the special characters from different languages aswell? — Marcus, Dec 01 '11 at 20:16
explaining why my result in a better approach being suggested. — , Dec 01 '11 at 20:19

score 26 · Accepted Answer · answered Dec 01 '11 at 20:19

26

There are the unicode character class thingys that you can use:

To match any non-letter symbols you can just use \PL+, the negation of \p{L}. To not remove spaces, use a charclass like [^\pL\s]+. Or really just remove punctuation with \pP+

Well, and obviously don't forget the regex /u modifier.

answered Dec 01 '11 at 20:19

mario

144,265
20
237
291

I think this one has potential. I need to investigate. Thanks mario – Thomas Dec 01 '11 at 20:30
to match space character (code point) use \pZ instead of \s – Mr Coder May 01 '15 at 08:22
1

This does not handle the non-numeric part. Correct answer should be `[^\pL^\pN\s]+` – genonymous Jul 03 '18 at 17:53

score 2 · Answer 2 · answered Mar 02 '14 at 10:16

2

I used this:

$clean = preg_replace( "/[^\p{L}|\p{N}]+/u", " ", $raw );
$clean = preg_replace( "/[\p{Z}]{2,}/u", " ", $clean );

answered Mar 02 '14 at 10:16

Олег Всильдеревьев

903
10
11

3

Can you perhaps describe what exactly is going on here? – Luke Rehmann Oct 26 '17 at 19:45

score 0 · Answer 3 · edited May 23 '17 at 12:01

0

Similar post

Remove non-utf8 characters from string

I'm not sure if this covers all characters though.

According to this post on th dreamincode forum

http://www.dreamincode.net/forums/topic/78179-regular-expression-to-remove-non-ascii-characters/

this should work

/[^\x{21}-\x{7E}\s\t\n\r]/

edited May 23 '17 at 12:01

Community

1
1

answered Dec 01 '11 at 20:19

CBusBus

2,321
1
18
26

I think this works only for ASCII, and the writer clearly mentiones UTF8 strings. – Pateman Dec 01 '11 at 20:22

score -2 · Answer 4 · answered Dec 01 '11 at 20:23

-2

Maybe this will be usefull?

$newstring = preg_replace('/[^0-9a-zA-Z\s]/', $oldstring);

answered Dec 01 '11 at 20:23

Scuba Kay

2,004
3
26
48

5

This will remove non ASCII characters as well. I don't want this – Thomas Dec 01 '11 at 20:27

Regex to remove non alphanumeric characters from UTF8 strings

4 Answers4

Linked

Related