Saving arabic characters into database using Rest and spring boot

Question

I have a rest web service that accepts a POST/PUT, the resource sent in the body of the rest call contains both French and Arabic characters , but when I send a resource that contain some Arabic characters, I get question marks stored in the database instead, as following : ?????.

When I type the Arabic characters manually in the database it's stored without any problem, this only happened when I use POST or PUT, in the other hand, The GET method works and it gets the Arabic characters.

This is the Request Headers source :

PUT /candidature/candidats HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 480
Accept: application/json, text/plain, */*
Origin: http://localhost:8080
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.87 Safari/537.36
Content-Type: application/json;charset=UTF-8
Referer: http://localhost:8080/app/index.html
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

As you can see in the Content-Type I have charset=UTF-8, so it should work in this case.

This is an example of a resource I'm sending in the PUT/POST method :

{"username":"ichigo","nom":"Kurusaki","prenom":"Ichigo","nomarab":"كوروساكي","prenomarab":"إيتشيغو","genre":"M"}

In this case after a POST/PUT the GET method will get the Arabic characters as question marks:

{"codeUser":null,"username":"Asus","nom":"sdf","prenom":"Mbarek","nomarab":"????????","prenomarab":"???????","genre":"M"}

But when I type them manually to the database and then I call the GET method, this is the result I'm getting :

{"codeUser":null,"username":"ichigo","nom":"Kurusaki","prenom":"Ichigo","nomarab":"كوروساكي","prenomarab":"إيتشيغو","genre":"M"}

So this problem happens only with the POST/PUT methods.

I'm using AngularJS in the client side and Spring boot in the server side.

So why I'm getting this behavior ? and how can I solve it ?

Edit :

In the server side I printed the Arabic values before I save them using Spring boot data as following :

System.out.print("arabicName : " + candidat.getNomarab());

and in console I got this :

arabicName : كوروساكي

So this happens when I call the saveAndFlush method of the Sring Boot Data JPA.

Does your database accept UTF-8 chars? If that isn't supporting them nothing wil help you. — M. Deinum, Apr 28 '16 at 09:14
@M.Deinum as I said `When I type the Arabic characters manually in the database it's stored without any problem` — Renaud is Not Bill Gates, Apr 28 '16 at 09:20
Show the connection properties and explain which database you are using. If the connection isn't configured to use UTF-8 then nothing will stored as such, you typing directly in the database are bypassing this. Basically it boils down to your datasource and hibernate configuration (and not so much Spring Boot or Spring Data). — M. Deinum, Apr 28 '16 at 09:33

score 1 · Accepted Answer · answered Apr 28 '16 at 10:41

As @M.Deinum mentioned this was a connection misconfig, so I configured the connection to use UTF-8 as following :

spring.datasource.url=jdbc:mysql://localhost:3306/db_myfme?useUnicode=yes&characterEncoding=UTF-8&characterSetResults=UTF-8

spring.jpa.properties.hibernate.connection.characterEncoding=utf-8
spring.jpa.properties.hibernate.connection.CharSet=utf-8
spring.jpa.properties.hibernate.connection.useUnicode=true

server.tomcat.uri-encoding=UTF-8

# HTTP encoding (HttpEncodingProperties)
spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true

I would expect the `hibernate.connection` ones to be useless... As it is not hibernate who is in control of the connections but the datasource. Those will only apply if you let hibernate create an internal datasource. — M. Deinum, Apr 28 '16 at 11:25

score 0 · Answer 2 · answered Jan 03 '19 at 10:42

You just need to convert an MySQL database characterset and collation to UTF-8. This will support all languages.

This commands for mysql version 5.5.3 and upper version. See for Reference

ALTER DATABASE DB_NAME CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE TABLE_NAME CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

and for older version

ALTER DATABASE DB_NAME CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE TABLE_NAME CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

Then change your Connection String in application.properties or application.yml file. Here For Me application.yml

 spring:
      datasource:
        driver-class-name: com.mysql.jdbc.Driver
        url: jdbc:mysql://localhost:3306/DB_NAME?useSSL=false&useUnicode=yes&characterEncoding=UTF-8&characterSetResults=UTF-8
        username: root
        password: Password

Saving arabic characters into database using Rest and spring boot

Edit :

2 Answers2