PHP method for stripping duplicate chars from a multibyte string?

Question

Arrrgh. Does anyone know how to create a function that's the multibyte character equivalent of the PHP count_chars($string, 3) command?

Such that it will return a list of ONLY ONE INSTANCE of each unique character. If that was English and we had

"aaabggxxyxzxxgggghq xcccxxxzxxyx"

It would return "abgh qxyz" (Note the space IS counted).

(The order isn't important in this case, can be anything).

If Japanese kanji (not sure browsers will all support this):

漢漢漢字漢字私私字私字漢字私漢字漢字私

And it will return just the 3 kanji used:

漢字私

It needs to work on any UTF-8 encoded string.

Answer 1

Hey Dave, you're never going to see this one coming.

php > $kanji = '漢漢漢字漢字私私字私字漢字私漢字漢字私';
php > $not_kanji = 'aaabcccbbc';
php > $pattern = '/(.)\1+/u';
php > echo preg_replace($pattern, '$1', $kanji);
漢字漢字私字私字漢字私漢字漢字私
php > echo preg_replace($pattern, '$1', $not_kanji);
abcbc

What, you thought I was going to use mb_substr again?

In regex-speak, it's looking for any one character, then one or more instances of that same character. The matched region is then replaced with the one character that matched.

The u modifier turns on UTF-8 mode in PCRE, in which it deals with UTF-8 sequences instead of 8-bit characters. As long as the string being processed is UTF-8 already and PCRE was compiled with Unicode support, this should work fine for you.

---

Hey, guess what!

$not_kanji = 'aaabbbbcdddbbbbccgggcdddeeedddaaaffff';
$l = mb_strlen($not_kanji);
$unique = array();
for($i = 0; $i < $l; $i++) {
    $char = mb_substr($not_kanji, $i, 1);
    if(!array_key_exists($char, $unique))
        $unique[$char] = 0;
    $unique[$char]++;
}
echo join('', array_keys($unique));

This uses the same general trick as the shuffle code. We grab the length of the string, then use mb_substr to extract it one character at a time. We then use that character as a key in an array. We're taking advantage of PHP's positional arrays: keys are sorted in the order that they are defined. Once we've gone through the string and identified all of the characters, we grab the keys and join'em back together in the same order that they appeared in the string. You also get a per-character character count from this technique.

This would have been much easier if there was such a thing as mb_str_split to go along with str_split.

(No Kanji example here, I'm experiencing a copy/paste bug.)

---

Here, try this on for size:

function mb_count_chars_kinda($input) {
    $l = mb_strlen($input);
    $unique = array();
    for($i = 0; $i < $l; $i++) {
        $char = mb_substr($input, $i, 1);
        if(!array_key_exists($char, $unique))
            $unique[$char] = 0;
        $unique[$char]++;
    }
    return $unique;
}

function mb_string_chars_diff($one, $two) {
    $left = array_keys(mb_count_chars_kinda($one));
    $right = array_keys(mb_count_chars_kinda($two));
    return array_diff($left, $right);
}

print_r(mb_string_chars_diff('aabbccddeeffgg', 'abcde'));
/* => 
Array
(
    [5] => f
    [6] => g
)
*/

You'll want to call this twice, the second time with the left string on the right, and the right string on the left. The output will be different -- array_diff just gives you the stuff in the left side that's missing from the right, so you have to do it twice to get the whole story.

Answer 2

$name = "My string";
$name_array = str_split($name);
$name_array_uniqued = array_unique($name_array);
print_r($name_array_uniqued);

Much easier. User str_split to turn the phrase into an array with each character as an element. Then use array_unique to remove duplicates. Pretty simple. Nothing complicated. I like it that way.

Answer 3

Please try to check the iconv_strlen PHP standard library function. Can't say about orient encodings, but it works fine for european and east europe languages. In any case it gives some freedom!