I can't capture a strange character with regular expressions

Question

I'm having problems to capture and filter out a strange character some data came with, which is causing JSON data which include it to not being correctly parsed. I don't know why, since it's not included in the white list I created with this regular expression:

$string = preg_replace('/[^\w\dñÑáéíóúÁÉÍÓÚüÜºª\-_\/\s\\<>,;:.*\[\]\(\)+?¿!&%@=]/', '', $string);

Testing regular expression on Regexr. As you will see, this strange character is not captured

This is how it is displayed in the browser:

And this is how it's displayed in Pluma (a Linux editor):

When I copy it and try to insert it in Google, for instance, nothing is inserted. Really strange. I've never bumped into any situation like this.

Any idea on how to handle it?

@MátéSolymosi I'm using a whitelist. Those characters which won't comply with it, should be replaced with an empty string. — Luis Martin, Jul 10 '17 at 09:43
Can you add the problematic character to the question text? I could not identify it based on the screenshot. — Mate Solymosi, Jul 10 '17 at 09:45
Have you tried simply copy and paste it into your Regex in an UTF8-Editor? — Bernhard, Jul 10 '17 at 09:46
Whoops, my bad, sorry. Didn't notice the `^` at the beginning... — Mate Solymosi, Jul 10 '17 at 09:49
@MátéSolymosi Click on the Regexr link. It will display what you see in the screen capture — Luis Martin, Jul 10 '17 at 09:53
@LuisMartin I was able to copy-paste the character to the whitelist, and then it matched it without problems. I am also able to Google it. — Mate Solymosi, Jul 10 '17 at 09:55
@MátéSolymosi True! I also could. The thing is I'm forced in this case to use code edited in ISO-8859-1, which doesn't cover that character. Anyway, if it's not in the white list, it shouldn't pass the validation. — Luis Martin, Jul 10 '17 at 10:25
@LuisMartin Ah, I see. You could try one of these techniques: https://stackoverflow.com/a/6058533/1242470 — Mate Solymosi, Jul 10 '17 at 10:32

Casimir et Hippolyte · Accepted Answer · 2017-07-10T11:47:59.097

This mysterious character isn't so mysterious but is difficult to display with your editor since it is the page-breaking control character: Form Feed (\x0C see the ascii table)

This character is contained in the \s character class, that's why your pattern doesn't match it.

A solution consists to remove the \s from your pattern and to replace it with an exhaustive list of allowed whitespace characters.

To make things easier you can already put the class \h (if supported) that contains all horizontal whitespaces. Then add the vertical whitespaces you want by hand.

Note that if you are working with the Windows-1252 code page, keep it under the eyes to be sure not to forget anything and to shorten the pattern using character ranges.

Thank you! Seems like you answered while I was writing and posting my own answer. I found out somewhere else. Anyway. It's fair I mark yours as the accepted, instead of mine. — Luis Martin, Jul 10 '17 at 11:38

score 1 · Answer 2 · answered Jul 10 '17 at 11:36

I got it!

Turns out this character represents a form feed

It's included in \s like white space, \t, \r or \n.

By being more specific, I achieved what I wanted. The new regex:

/[^\w\dñÑáéíóúÁÉÍÓÚüÜºª\-_\/ \r\n\\<>,;:.*\[\]\(\)+?¿!&%@=]/

I can't capture a strange character with regular expressions

2 Answers2