2

I have a REST API endpoint that lets you upload a file to the server. When I save a file that is named using the latin alphabet there is no issue. But when I try to save a file that has a japanese character, the file is saved and the filename in the server is okay but when I look into the database the filename is not right.

The result in my DB when saving the file:

phpMyAdmin displays one rowset with text of nonsensical characters

But when I look at the server's shell the filename is correct:

puTTY displays a proper UTF-8 filename with Hiragana in a shell

I tried changing the database collation into:

  • utf8mb4_unicode_ci
  • utf8
  • utf8_general_ci ...but the issue still persists.

Update: This is my query to insert the data:

public function setFileQuery($param,$file){
  $this->module_id = $param['moduleid'];
  $this->file_name = $file['name'];
  $this->file_size = $file['size'];

  $q = "INSERT INTO data_file
       (module_id,file_name,file_size,start_time,end_time,elapse_time,through_put)
       VALUES
       (?,?,?,?,?,?,?)";
            
  $insertStmt = $this->conn->prepare($q);
  $insertStmt->execute([
    $this->module_id,
    $this->file_name,
    $this->file_size,
    null,
    null,
    null,
    null,
  ]);
            
  Responses::http_ok();
}
AmigoJack
  • 5,234
  • 1
  • 15
  • 31
draw134
  • 1,053
  • 4
  • 35
  • 84
  • 1
    Can you share the ``insert`` query code which you use to insert data to the table? – Dula Dec 01 '21 at 03:32
  • Done editng the question sir @Dula – draw134 Dec 01 '21 at 03:36
  • See "Mojibake" in https://stackoverflow.com/questions/38363566/trouble-with-utf8-characters-what-i-see-is-not-what-i-stored The probably is probably with the connection parameters, but I don't seen enough details to say for sure. – Rick James Dec 02 '21 at 19:02

1 Answers1

3
  1. First, the table collation should be set utf8mb4.

  2. Second, make sure that the data field (aka column) collations are also utf8mb4.

  3. Now you may insert data (e.g. thru phpMyAdmin or thru any PHP script):

    • phpMyAdmin: phpMyAdmin INSERT query with Hiragana literal

    • PHP:

      • If you are using PDO then make sure you have set the charset in your connection, such as:
        $conn = new PDO("mysql:host=$host;dbname=$db;charset=utf8mb4", $user, $pass);
        
      • If you are using MySQLi:
        $conn = mysqli_connect($servername, $username, $password, $dbname);
        
        /* change character set to utf8mb4 */
        mysqli_set_charset($conn,"utf8mb4");
        $mysqli->query("set names utf8mb4");
        

      ...and then executing the query, i.e.:

      mysqli_query( "INSERT INTO song( songname ) VALUES( 'あああ.txt' )" );
      
  4. The result will be as wanted: phpMyAdmin displaying one rowset with Hiragana text

Ken Lee
  • 6,985
  • 3
  • 10
  • 29
  • does changing collation directly into phpmyadmin table different? – draw134 Dec 01 '21 at 03:58
  • Phpmyadmin is just a tool (but it is quite handy) for performing db operation , such as changing the db structure (including collation). Or you may enter command thru the mysql console too. They will both be ok. For PHP, just make sure you have indicated that you are using utf8 as charset (see my example lines in PDO and Mysqli) – Ken Lee Dec 01 '21 at 04:08
  • Please don't use MyISAM; it is going away. It may mislead MySQL users. Also, please use utf8mb4, not utf8; the former is being standardized. (Neither of these comments impact the question about mangled Japanese characters.) – Rick James Dec 02 '21 at 19:04
  • @RickJames, Thanks for your comments, I have amended my answer by removing MyISAM and changed all utf8 to utf8mb4. – Ken Lee Dec 03 '21 at 01:39