How to remove "ï¿½" from php string?

Question

Possible Duplicate:
How to replace ï¿½ in a string

I am reading data from an XML sheet coming out of a database. In the raw output I am coming accross this character "ï¿½" which is the UTF-8 string meaning "�". Doing a simple search and remove with str_replace does not do the trick when searching for "�" or "ï¿½". Is there any other way to remove this character from a string?

UPDATE:

For reference this is the function that is cleaning up strings for me.

    function db_utf8_convert($str)
{
    $convmap = array(0x80, 0x10ffff, 0, 0xffffff);
    return preg_replace('/\x{EF}\x{BF}\x{BD}/u', '', mb_encode_numericentity($str, $convmap, "UTF-8"));
}

http://stackoverflow.com/search?q=remove+bom+%5Bphp%5D&submit=search — c69, Dec 27 '12 at 19:53
1st result is a SO question, answered: https://www.google.com/search?q=ï¿½ Not sure if that helps or not. — The111, Dec 27 '12 at 19:54
tried that with a php equivalent statement before asking this, didn't work. — labago, Dec 27 '12 at 19:56
That's the UTF-8 BOM. The � means invalid character. If you used a parser then you'd not have this problem. — David Heffernan, Dec 27 '12 at 19:56

PhearOfRayne · Answer 1 · 2012-12-27T20:37:22.320

3

You can do this:

$str = 'UTF-8 string meaning "ï¿½"';
echo preg_replace('/\x{EF}\x{BF}\x{BD}/u', '', iconv(mb_detect_encoding($str), 'UTF-8', $str));

Output: UTF-8 string meaning ""

edited Dec 27 '12 at 20:37

answered Dec 27 '12 at 20:21

PhearOfRayne

4,990
3
31
44

I had high hopes for this but it sadly is not working for me. – labago Dec 27 '12 at 20:28
1

@jlane09 If my first answer was not working it is due to the fact your server is using an encoding other than UTF-8, so I updated my answer for you. – PhearOfRayne Dec 27 '12 at 20:39
Still didnt want to work, I appreciate the continued effort though. – labago Dec 27 '12 at 20:46
Yeah I figured it might not, since `mb_detect_encoding()` is very limited! Any reason you didn't set your encoding on your DB and Server to UTF-8? This would prevent a lot of future and current issues. – PhearOfRayne Dec 27 '12 at 20:49
The DB I am inserting these strings into is encoded in UTF-16. The DB it is coming out of I am not sure and have no control over. The server it is running on is UTF-8 – labago Dec 27 '12 at 20:52
You should be using one or the other not both. – PhearOfRayne Dec 27 '12 at 21:02

Get Off My Lawn · Answer 2 · 2012-12-27T20:20:46.877

2

You could do something similar to this:

<?php
$string = "asd fsa fsaf sf ï¿½ asdfasdfs";

echo preg_replace("/[^\p{Latin} ]/u", "", $string);

Check out this script for more character matches:
http://www.regular-expressions.info/unicode.html#script

EDIT

I did find, this, people says it works, you could give it a try:

<?php
function removeBOM($str=""){
    if(substr($str, 0,3) == pack("CCC",0xef,0xbb,0xbf)) {
        $str=substr($str, 3);
    }
    return $str;
}
?>

edited Dec 27 '12 at 20:20

answered Dec 27 '12 at 20:06

Get Off My Lawn

34,175
38
176
338

1

That'll also remove many other characters. – rid Dec 27 '12 at 20:08
That is why I added the link, it has more combos in it. – Get Off My Lawn Dec 27 '12 at 20:10
Indeed, but any of the listed scripts will remove *all* characters that are part of that script, not just these 3. – rid Dec 27 '12 at 20:11
If someone could post a preg_match that only removes the BOM that would be spectacular. – labago Dec 27 '12 at 20:14
1

Nope, those little BOM's are still popping up using that function. – labago Dec 27 '12 at 20:24

How to remove "ï¿½" from php string?

2 Answers2