PHP: checking if string is utf-8 or utf-16le

Question

I'm writing function that returns document in utf-8 by default, by if you provide special parameter it will return content in utf-16le. I started to write unit test for this function and what I don't understand is how to check if encoding is "utf-8" or "utf-16le". I've tried mb_detect_encoding but it returns false even in this code snippet:

  $utf16Doc = mb_convert_encoding($doc, "utf-16le", "utf8");
  $test = mb_detect_encoding($utf16Doc, "utf-16le");
  var_dump($test);

Appreciate any ideas how to check in test that encoding is utf-16le and not utf-8.

You shouldn't need to detect the encoding, just check that it is what the test case expects. — Tom Blodget, Mar 30 '18 at 01:24

A l w a y s S u n n y · Answer 1 · 2018-03-29T01:58:49.897

A simple way to detect UTF-8/16/32 of file by its BOM (not work with string or file without BOM)

<?php
// Unicode BOM is U+FEFF, but after encoded, it will look like this.
define ('UTF32_BIG_ENDIAN_BOM'   , chr(0x00) . chr(0x00) . chr(0xFE) . chr(0xFF));
define ('UTF32_LITTLE_ENDIAN_BOM', chr(0xFF) . chr(0xFE) . chr(0x00) . chr(0x00));
define ('UTF16_BIG_ENDIAN_BOM'   , chr(0xFE) . chr(0xFF));
define ('UTF16_LITTLE_ENDIAN_BOM', chr(0xFF) . chr(0xFE));
define ('UTF8_BOM'               , chr(0xEF) . chr(0xBB) . chr(0xBF));

function detect_utf_encoding($filename) {

    $text = file_get_contents($filename);
    $first2 = substr($text, 0, 2);
    $first3 = substr($text, 0, 3);
    $first4 = substr($text, 0, 3);

    if ($first3 == UTF8_BOM) return 'UTF-8';
    elseif ($first4 == UTF32_BIG_ENDIAN_BOM) return 'UTF-32BE';
    elseif ($first4 == UTF32_LITTLE_ENDIAN_BOM) return 'UTF-32LE';
    elseif ($first2 == UTF16_BIG_ENDIAN_BOM) return 'UTF-16BE';
    elseif ($first2 == UTF16_LITTLE_ENDIAN_BOM) return 'UTF-16LE';
}
?>

So now you can try like this,

 $utf16Doc = mb_convert_encoding($doc, "utf-16le", "utf8");
 $test = detect_utf_encoding($utf16Doc);
 var_dump($test);

Clearly Stated : This is not my own written answer. I just found it on php.net official site under mb_detect_encoding by a user http://php.net/manual/en/function.mb-detect-encoding.php#68607 . Hope this will help you somehow

I understand, that this is not your answer, but shouldn't it be `$first4 = substr($text, 0, 4)` instead of using 3 bytes? — rabudde, Mar 06 '23 at 10:34

PHP: checking if string is utf-8 or utf-16le

1 Answers1

Linked