Streaming JSON directly to response with Jackson

Question

Currently, I need to send a large json object to an ajax request. For the purpose I am using the following controller method which works fine.

   @RequestMapping(method = RequestMethod.POST,params = {"dynamicScenario"})
   @ResponseBody
    public String getDynamicScenarioData(@RequestParam Map<String, String> map) throws JsonParseException, JsonMappingException, IOException
 {

  ObjectMapper mapper = new ObjectMapper();

    @SuppressWarnings("unchecked")
    Map<String,Object> queryParameters = mapper.readValue(map.get("parameters") , Map.class);

    Map<String, Object> getData = service.runDynamicScenario(queryParameters, map.get("queryString"));

    return writer.writeValueAsString(getData); //here java throws java.lang.OutOfMemoryError: Java heap space memory
} 

Update: my ajax is:

          $.ajax({
          type: "POST",
          url: "dynamicScenario.htm",
          data : tags,
          dataType: "json",
          success: function(data){});

My DispatcherServlet settings:

           public class ApplicationInitializer implements      WebApplicationInitializer
      {
       public void onStartup(ServletContext servletContext) throws      
     ServletException 
     {
    AnnotationConfigWebApplicationContext context = new AnnotationConfigWebApplicationContext();
    context.register(ApplicationConfig.class);

    servletContext.addListener(new ContextLoaderListener(context));

    ServletRegistration.Dynamic servletRegistration = servletContext.addServlet("dispatcher", new DispatcherServlet(context));
    servletRegistration.setLoadOnStartup(1);
    servletRegistration.addMapping("*.htmlx");
  }
}

I am using jackson to serialize a map of different objects and then send it back to the ajax. However, if the size of the json is large then java throws out of memory. I know the Jackson method writer.writeValueAsString is inefficient because its writing to a string but is there any other alternative? I can't use plain java POJO because I don't know what objects the map which I will have to serialize will contain so I can't simply map it to some java object. Any ideas? Thanks

Answer 1

The issue you want to solve is

return writer.writeValueAsString(getData); 

creating too big of a String and causing an OutOfMemoryError. Jackson supports a Streaming API, which Spring makes use of in its MappingJackson2HttpMessageConverter, which handles serializing POJOs to JSON in the response body (and request body).

There are a few ways to handle this. The easiest is to change your return type to Map<String, Object and return the corresponding object directly.

@RequestMapping(method = RequestMethod.POST,params = {"dynamicScenario"})
@ResponseBody
public Map<String, Object> getDynamicScenarioData(@RequestParam Map<String, String> map) throws JsonParseException, JsonMappingException, IOException
 {
     ObjectMapper mapper = new ObjectMapper();

     @SuppressWarnings("unchecked")
     Map<String,Object> queryParameters = mapper.readValue(map.get("parameters") , Map.class);

     Map<String, Object> getData = service.runDynamicScenario(queryParameters, map.get("queryString"));

     return getData;
} 

Spring will take care of serializing getData by streaming the results directly to the response OutputStream.

This won't work on its own. For one, the url you are using to access your service

/dynamicScenario.htm

is causing Spring's content negotiation to kick in. It sees .htm and thinks you are expecting text/html content. Either turn off content negotiation or use a URL without an extension

/dynamicScenario

Spring will then, instead, look at the Accept headers for figuring out what your client is expecting. Since that is

dataType: "json"

it's going to write application/json with the MappingJackson2HttpMessageConverter.

---

Here are some more things to fix

AnnotationConfigWebApplicationContext context = new AnnotationConfigWebApplicationContext();
context.register(ApplicationConfig.class);

servletContext.addListener(new ContextLoaderListener(context));

ServletRegistration.Dynamic servletRegistration = servletContext.addServlet("dispatcher", new DispatcherServlet(context));
servletRegistration.setLoadOnStartup(1);
servletRegistration.addMapping("*.htmlx");

You're currently making both the ContextLoaderListener and DispatcherServlet load the same ApplicationContext. This is unnecessary and potentially harmful. The DispatcherServlet already uses the ContextLoaderListener context as a parent context. You can read more about it here:

• What is the difference between ApplicationContext and WebApplicationContext in Spring MVC?

If your ApplicationConfig only contains MVC configuration elements, only have the DispatcherServlet load it. You won't need a ContextLoaderListener.

If it has other types of config elements unrelated to the MVC stack, split it into two @Configuration classes. Pass the MVC one to the DispatcherServlet and the other to the ContextLoaderListener.

Don't pass JSON as form parameters. Pass it directly into the request body and use @RequestBody to deserialize it directly into your expected type.

@RequestMapping(method = RequestMethod.POST)
@ResponseBody
public Map<String, Object> getDynamicScenarioData(@RequestBody Map<String,Object> queryParameters) throws JsonParseException, JsonMappingException, IOException
 {
     Map<String, Object> getData = service.runDynamicScenario(queryParameters, /* find a better way to pass this map.get("queryString") */);

     return getData;
}

You save yourself an ObjectMapper object on each request (it's a heavy object) and you let Spring take care of all the deserializing (again streaming). You'll have to find another way to pass the queryString (you can still pass it as a single query parameter and get it with `@RequestParam).