ObjectMapper Not Handling UTF-8 Properly?

Question

I'm using ObjectMapper to serialize posts in my system to json. These posts contain entries from all over the world and contain utf-8 characters. The problem is that the ObjectMapper doesn't seem to be handling these characters properly. For example, the string "Musée d'Orsay" gets serialized as "Mus?©e d'Orsay".

Here's my code that's doing the serialization:

public static String toJson(List<Post> posts) {
        ObjectMapper objectMapper = new ObjectMapper()
            .configure(Feature.USE_ANNOTATIONS, true);
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        try {
            objectMapper.writeValue(out, posts);
        } catch (JsonGenerationException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (JsonMappingException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        return new String(out.toByteArray());
    }

Interestingly, the exact same List<Post> posts gets serialized just fine when I return it via a request handler using @ResponseBody using the following configuration:

public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
    ObjectMapper m = new ObjectMapper()
        .enable(Feature.USE_ANNOTATIONS)
        .disable(Feature.FAIL_ON_UNKNOWN_PROPERTIES);
    MappingJacksonHttpMessageConverter c = new MappingJacksonHttpMessageConverter();
    c.setObjectMapper(m);
    converters.add(c);
    super.configureMessageConverters(converters);
}

Any help greatly appreciated!

Is it your controller method which returns String directly? http://stackoverflow.com/questions/3616359/who-sets-response-content-type-in-spring-mvc-responsebody — Boris Treukhov, Aug 11 '12 at 08:02
I think you should update that question, as the problem is NOT in ObjectMapper. — philipp, May 06 '14 at 19:29

score 2 · Accepted Answer · answered Aug 11 '12 at 22:30

2

Aside from conversions, how about simplifying the process to:

return objectMapper.writeValueAsString(posts);

which speeds up the process (no need to go from POJO to byte to array to decode to char to build String) as well as (more importantly) shortens code.

answered Aug 11 '12 at 22:30

StaxMan

113,358
34
211
239

Perfect. So much simpler than what I was doing. Thanks!! – threejeez Aug 11 '12 at 23:21

score 1 · Answer 2 · answered Aug 11 '12 at 08:12

1

Not 10 minutes later and I found the problem. The issue wasn't with the ObjectMapper, it was with how I was turning the ByteArrayOutputStream into a string. I changed the code as follows and everything started working:

try {
        return out.toString("utf-8");
    } catch (UnsupportedEncodingException e) {
        return out.toString();
    }

answered Aug 11 '12 at 08:12

threejeez

2,314
6
30
51

Disregard this. While it works @StaxMan's answer is far better. – threejeez Aug 11 '12 at 23:23

ObjectMapper Not Handling UTF-8 Properly?

2 Answers2

Linked