2

Even when i type 

echo strtr("-äåö-", "äåö", "xxx");

it does not work properly it outputs this >xxx¥x¶<, however when i use example below it does not translate nothing at all it keeps original mambo jumbo. If i type in form ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÑŐŰÜÒÓÔÕÖØÝߟàáâãäåçèéêëìíîïñòóôõőöøšűùúûüýÿž and click translate it outpputs the same string and æ œ does not translate at all.

<form method="POST">
    <input style="width:500px;" type="text" name="first_name" />
    <input style="width:500px;" type="text" name="last_name" />
    <input type="submit" name="submit" value="translate" />
</form>


<?php

    $dict = array(
                    "Æ" => "AE", 
                    "æ" => "ae", 
                    "Œ" => "OE",
                    "œ" => "oe"
                 );

    $first = strtr($_POST['first_name'], $dict);         
    $last  = strtr($_POST['last_name'], $dict);      


    $first = strtr($first, 
                          "ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÑŐŰÜÒÓÔÕÖØÝߟàáâãäåçèéêëìíîïñòóôõőöøšűùúûüýÿž",
                          "AAAAAACEEEEIIIINOUUOOOOOOYSYaaaaaaceeeeiiiinooooooosuuuuuyyz"); 

    $last  = strtr($last, 
                          "ÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÑŐŰÜÒÓÔÕÖØÝߟàáâãäåçèéêëìíîïñòóôõőöøšűùúûüýÿž",
                          "AAAAAACEEEEIIIINOUUOOOOOOYSYaaaaaaceeeeiiiinooooooosuuuuuyyz"); 

    echo $first." --- ";
    echo $last;
?>

even when i added on top of code

foreach ($_POST as $key => $value) {
    $POST[$key] = iconv(mb_detect_encoding($_POST["first_name"]), "ASCII//TRANSLIT", $POST[$value]);
}

and paste AAAAAACEEEEIIIINOUUOOOOOOYSYaaaaaaceeeeiiiinooooooosuuuuuyyz it comes out like this yAyAyAyEyEyIyIyNyUyOyOyOyYyYyayauaueyeyiyiynyoyoyoysyuuuyyyzy�y�y�y�y�y�y�y�y�y�y�y�y�y�y�y�y�y�uay�yuuzu�y�y�y�y�y�y�u�

Never mind since no one knows why it does not work i just used str_replace and str_ireplace very successfully without need for any concerns about encoding.

EDIT: my bad encoding does matter for str_replace too. i used on html page

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
JohnA
  • 1,875
  • 8
  • 28
  • 34
  • 3
    Do you mind if I ask why you feel it necessary to replace valid UTF-8 characters with ASCII? – Borealid Mar 14 '12 at 23:14
  • i dont much about character encoding. so if i got äåö it wont be UTF-8 what encoding will it be? – JohnA Mar 15 '12 at 00:27

3 Answers3

3

strtr with the function prototype

string strtr ( string $str , string $from , string $to )

does only work reliable with single-byte encodings (e.g. ISO-8859-1).

header("Content-Type: text/plain; charset=ISO-8859-1");
$str = "\x2d\xe4\xe5\xf6\x2d"; // ISO-8859-1: -äåö-
$from = "\xe4\xe5\xf6";        // ISO-8859-1: äåö
$to = "\x78\x78\x78";          // ISO-8859-1: xxx
dump($str, "ISO-8859-1");  // length in octets: 5
dump($from, "ISO-8859-1"); // length in octets: 3
dump($to, "ISO-8859-1");   // length in octets: 3

print strtr($str, $from, $to); // -xxx-

Output:

-: 2d
ä: e4
å: e5
ö: f6
-: 2d
length (encoding: ISO-8859-1): 5
length in octets (8-bit-byte): 5

ä: e4
å: e5
ö: f6
length (encoding: ISO-8859-1): 3
length in octets (8-bit-byte): 3

x: 78
x: 78
x: 78
length (encoding: ISO-8859-1): 3
length in octets (8-bit-byte): 3

-xxx-

If you use a multi-byte chars e.g. from UTF-8 you'll likely get a messed up string:

header("Content-Type: text/plain; charset=UTF-8");
$str = "\x2d\xc3\xa4\xc3\xa5\xc3\xb6\x2d"; // UTF-8: -äåö-
$from = "\xc3\xa4\xc3\xa5\xc3\xb6";        // UTF-8: äåö
$to = "\x78\x78\x78";                      // UTF-8: xxx
dump($str, "UTF-8");  // length in octets: 8
dump($from, "UTF-8"); // length in octets: 6
dump($to, "UTF-8");   // length in octets: 3

// > If from and to have different lengths, the extra characters in the longer
// > of the two are ignored. The length of str will be the same as the return
// > value's.
// http://de.php.net/manual/en/function.strtr.php

// This means that the $from-string gets cropped to "\xc3\xa4\xc3" (16 bit of
// the first char [ä] and the first 8 bit of the second char [å]):
strtr($str, $from, $to) === strtr($str, "\xc3\xa4\xc3", $to); // true
print strtr($str, $from, $to); // -xxx�x�-

Output:

-: 2d
ä: c3a4
å: c3a5
ö: c3b6
-: 2d
length (encoding: UTF-8): 5
length in octets (8-bit-byte): 8

ä: c3a4
å: c3a5
ö: c3b6
length (encoding: UTF-8): 3
length in octets (8-bit-byte): 6

x: 78
x: 78
x: 78
length (encoding: UTF-8): 3
length in octets (8-bit-byte): 3

-xxx�x�-

For multibyte encodings like UTF-8 you have to use the second function prototype:

string strtr ( string $str , array $replace_pairs )

header("Content-Type: text/plain");
$str = "-äåö-"; // UTF-8 \x2d\xc3\xa4\xc3\xa5\xc3\xb6\x2d
$replace_pairs = array(
    "ä" /* UTF-8 \xc3\xa4 */ => "x",
    "å" /* UTF-8 \xc3\xa5 */ => "x",
    "ö" /* UTF-8 \xc3\xb6 */ => "x"
);
print strtr($str, $replace_pairs); // -xxx-

If the encodings do not match you have to convert them with iconv:

header("Content-Type: text/plain");
$str = "\x2d\xe4\xe5\xf6\x2d"; // ISO-8859-1 -äåö-
$str = iconv("ISO-8859-1", "UTF-8", $str);
$replace_pairs = array(
    "ä" /* UTF-8 \xc3\xa4 */ => "x",
    "å" /* UTF-8 \xc3\xa5 */ => "x",
    "ö" /* UTF-8 \xc3\xb6 */ => "x"
);
print strtr($str, $replace_pairs); // -xxx-

function dump:

// outputs the hexvalue for each char for the given encoding
function dump($data, $encoding) {
    for($i = 0, $len = iconv_strlen($data, $encoding); $i < $len; ++$i) {
        $char = iconv_substr($data, $i, 1, $encoding);
        printf("%s: %s\n", $char, bin2hex($char));
    }
    printf("length (encoding: %s): %d\n", $encoding, $len);
    printf("length in octets (8-bit-byte): %d\n\n", strlen($data));
}
Saxoier
  • 1,287
  • 1
  • 8
  • 8
1

Did you try mb_strstr: http://php.net/manual/en/function.mb-strstr.php

This function supports Multibyte character encoding.

stan
  • 4,885
  • 5
  • 49
  • 72
1

It sounds like you may have competing encodings. If your browser is submitting UTF8 but your file is saved in (e.g.) 8859-1, your characters will not match and the translate will fail. In addition, looking at the doc page, several comments suggest using utf8_decode() on your input string first. It is possible that utf8_decode() by itself will do what you want.

UTF8 is a multi-byte encoding (actually, it's a variable-byte encoding). Characters such as ÷ or ï have a Unicode code point beyond 256 which needs to be encoded into two or more bytes, both above 128, that identify the character. I suspect you are going to have to learn more about Unicode. There is another explanation at utf8_encode.

Edit: It's been a while since I've wrestled with encodings. You should look at iconv() for more general purpose re-encoding.

staticsan
  • 29,935
  • 4
  • 60
  • 73