0

I got a really weird problem. I got two same string which is chinse "君山", one from kafka ,and one from a mysql field(utf8mb4). I write two strings into another table of mysql,also the field is utf8mb4 encoded, the one from kafka turns into unreadeble code "??",the other is ok!

Then i print two string with follow java code

//old_name from kafka
//group.getName() from mysql
//old_name,group.getName() should be the same "君山"
char[] oldNameCharArray = old_name.toCharArray();
char[] newNameCharArray = group.getName().toCharArray();

System.out.print("oldName:")
for(char ch : oldNameCharArray) {
    int value = (int)ch;
    System.out.print((Integer.toHexString(value)));
}
System.out.println("-----------------------------------------");
System.out.print("newName:");
for(char ch : newNameCharArray) {
    int value = (int)ch;
    System.out.print(Integer.toHexString(value));
}

I got output like below:

oldName:541b5c71
-----------------------------------------
newName:15a17af515a12a12a

I have check the unicode table, chinse text "君" should be "0x541b" and "山" should be "0x5c71". So the oldName make sense, i don't know what kind of code is for the newName output, and more hard to understand is that the oldName turns into unreadable code "??" after i save it to mysql,whild the newName is good.

By the way, i use spring + mybatis framework to operate mysql

dzthink
  • 33
  • 5
  • Have you checked the character set of your connection? What would the command `show variables like 'character_set%';` show? – leeyuiwah Jan 07 '17 at 12:59
  • yes, i have checked that, i use set names utf8mb4 for each connection – dzthink Jan 07 '17 at 13:09
  • UTF8 is not the same as UTF8mb4. These two links may be helpful (1) http://stackoverflow.com/questions/5078314/isnt-the-size-of-character-in-java-2-bytes (2) https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html – leeyuiwah Jan 07 '17 at 13:40
  • `541b` is Unicode, not utf8/utf8mb4. How was it generated? Unicode and UTF-8 are related, but different. – Rick James Jan 07 '17 at 19:04
  • It was generated by android client,transfered to server through http api implemented bt php,php serialize it by json and then put it into kafka,java consumer in th question get it from kafka and write it into mysql – dzthink Jan 08 '17 at 05:37

2 Answers2

0

I sovled the problem by chance,my mysql server is a rds service from alibaba cloud service. I changed my jdbc connector string and the problem is sovled, and i don't know why!

enter image description here

The one above is old one, and the one below solved my problem

my jdbc conntector version is 5.1.31

dzthink
  • 33
  • 5
-1

just change system locale of your server machine in control panel.

here a video is available that show how to change it.

mehdi
  • 912
  • 7
  • 20