Error returning UTF-8 character as Json Array in Springs MVC

Question

I am facing a problem in returning certain UTF-8 Character Strings which are in my Google Cloud SQL Database. I am sharing my code below.

public JSONArray getObjectsJsonArrayByCountryCodeAndDestination(String countryCode, String destinationCountryStartingLetter)
        throws JSONException, UnsupportedEncodingException {
    String query = "select * from EMBASSY where COUNTRY_CODE = ? AND DESTINATION LIKE '"+destinationCountryStartingLetter+"%' order by DESTINATION";
    jdbcTemplate = new JdbcTemplate(dataSource);
    List<Embassy> list = jdbcTemplate.query(query,
            new Object[] { countryCode }, new EmbassyMapper());
    JSONArray jsonArray = new JSONArray();
    if (list.isEmpty()) {
        return jsonArray;
    } else {
        for (Embassy embassy : list) {
            String address = new String(embassy.getAddress().getBytes("UTF-8"), "UTF-8");
// Upto here I am getting correct value of Address
            JSONObject jsonObject = new JSONObject();
            jsonObject.put("ID", embassy.getID());
            jsonObject.put("type", embassy.getType());
            jsonObject.put("telephone", embassy.getTelephone());
            jsonObject.put("address", address);
            jsonObject.put("url", embassy.getUrl());
            jsonObject.put("destination", embassy.getDestination());
            jsonObject.put("status", embassy.getStatus());
            jsonObject.put("updatedOn", embassy.getUpdatedOn());
            jsonObject.put("countryCode", embassy.getCountryCode());
            //System.out.println(jsonObject.toString());
            jsonArray.put(jsonObject);
        }
        return jsonArray;
    }
}

I have all correct encoded records saved in my database but when I am returning these in the form of JSONArray to my Ajax Request I am getting certain UTF-8 characters changed. like I got Espaňa as Espa?a why is its so.

I am using json.org library for JSON generation.

For clarity, where does the encoding go wrong? Database reads/HTTP transmission/...? — Joachim Isaksson, Jan 10 '13 at 11:21
when I am putting the address value in JSONObject. I debugged my code and I am getting correct value in String address when I am encypting it to UTF-8. After that it is not putting the correct value. — Ankur Jain, Jan 10 '13 at 11:23
I have no idea how to do it for Spring/MVC, but since that's based on top of Servlet API, all I can tell you is that you somehow need to make sure that it's calling `response.setCharacterEncoding("UTF-8")` before writing any byte/char to the response output stream. Otherwise it will use platform default character encoding which is often ISO-8859-1. Any character which is not covered by that charset would otherwise be replaced by a question mark which matches exactly your symptoms. — BalusC, Jan 10 '13 at 18:25
Why `String address = new String(embassy.getAddress().getBytes("UTF-8"), "UTF-8");` instead of `String address = embassy.getAddress();` — Henry, Jan 10 '13 at 18:27
@Henry: it's likely result of clueless shooting in the dark or having smoked a bad joint. It does not do anything useful expect that it's more sensitive to a `NullPointerException`. — BalusC, Jan 10 '13 at 18:28
Perhaps you called it too late? The response should not be committed at that point. It's otherwise plain ignored. Or perhaps it's overridden by Spring/MVC itself later on? Not sure as I don't use it. — BalusC, Jan 10 '13 at 18:31
@BalusC I have read in one of your comments that you recommend GSON library. And the major error where I am getting this error when I try to print it into the console or I put it in the JSONObject. — Ankur Jain, Jan 10 '13 at 18:34
Getting it as such in the console (via `System.out`) has a different cause. It has to be configured in the side of the owner of the console. If it's for example Eclipse, then see also http://stackoverflow.com/questions/7539590/can-not-send-special-characters-utf-8-from-jsp-to-servlet-question-marks-disp/7539605#7539605. — BalusC, Jan 10 '13 at 18:37
Particularly what I need is to get a JSON Array in response of my AJAX call which prevents the UTF-8 encoded characters in it. — Ankur Jain, Jan 10 '13 at 18:44

score 0 · Accepted Answer · answered Jan 10 '13 at 19:30

Finally I have solved this problem with the contribution of you people comments. I have added this few lines in the ResponseBody of my spring controller method and I have also made these changes in my JSP: I am sharing below my full code.

JSP

<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

Dao Implementation for Retrieval

public JSONArray getObjectsJsonArrayByCountryCodeAndDestination(String countryCode, String destinationCountryStartingLetter)
        throws JSONException, UnsupportedEncodingException {
    String query = "select * from EMBASSY where COUNTRY_CODE = ? AND DESTINATION LIKE '"+destinationCountryStartingLetter+"%' order by DESTINATION";
    jdbcTemplate = new JdbcTemplate(dataSource);
    List<Embassy> list = jdbcTemplate.query(query,
            new Object[] { countryCode }, new EmbassyMapper());
    JSONArray jsonArray = new JSONArray();
    if (list.isEmpty()) {
        return jsonArray;
    } else {
        for (Embassy embassy : list) {
            String address = new String(embassy.getAddress().getBytes("UTF-8"), "UTF-8");
            String address2 = embassy.getAddress();
            System.out.println(address+" : "+address2);
            JSONObject jsonObject = new JSONObject();
            jsonObject.put("ID", embassy.getID());
            jsonObject.put("type", embassy.getType());
            jsonObject.put("telephone", embassy.getTelephone());
            jsonObject.put("address", address);
            jsonObject.put("url", embassy.getUrl());
            jsonObject.put("destination", embassy.getDestination());
            jsonObject.put("status", embassy.getStatus());
            jsonObject.put("updatedOn", embassy.getUpdatedOn());
            jsonObject.put("countryCode", embassy.getCountryCode());
            System.out.println(jsonObject.toString());
            jsonArray.put(jsonObject);
        }
        return jsonArray;
    }
}

Spring Controller Method

@RequestMapping(value = "/admin/listembassy", method = RequestMethod.GET)
public @ResponseBody void listEmbassyByCountryCode(HttpServletRequest req,HttpServletResponse resp, ModelMap modelMap) throws JSONException, JsonGenerationException, JsonMappingException, IOException {
    String countryCode = req.getParameter("searchForCountryCode");
    String destinationCountryStartLetter = req.getParameter("destinationCountryStartLetter");
    EmbassyDao embassyDao = (EmbassyDao) context.getBean("embassyDao");
    JSONArray jsonArray = new JSONArray();
    if(destinationCountryStartLetter != null && !destinationCountryStartLetter.equals("")){
        jsonArray = embassyDao.getObjectsJsonArrayByCountryCodeAndDestination(countryCode, destinationCountryStartLetter);
    }else{
        jsonArray = embassyDao.getObjectsJsonArrayByCountryCode(countryCode);
    }
    resp.setContentType("application/json");
    resp.setCharacterEncoding("UTF-8");
    resp.getWriter().println(jsonArray);
}

I wonder how JSP encoding is relevant to the concrete problem of writing question marks. Does Spring forward to a JSP after having written the JSON? Wouldn't that corrupt the JSON with trailing HTML content of the JSP? Essentially, all you seem to have done in order to fix the problem is just adding `resp.setCharacterEncoding("UTF-8")`, right? — BalusC, Jan 11 '13 at 03:06
Sir I just don't want to take any risk so I have seen this in a post in stackoverflow only and I followed the same way. — Ankur Jain, Jan 11 '13 at 05:20
Okay, but you have commented that setting the response encoding didn't work for you. — BalusC, Jan 11 '13 at 11:02
@BalusC Sorry for that sir, I have removed that comment so that others didn't misunderstood if they get the same error or problem. But initially it was really not working. Initially I was returning a JSONArray as string from my controller and I was parsing that string on my ajax, but when I changed it to **resp.getWriter().println(jsonArray)** it worked like a charm. — Ankur Jain, Jan 11 '13 at 11:27

Error returning UTF-8 character as Json Array in Springs MVC

1 Answers1