0

I'm working on a Java server with a JSON REST API powered by Jersey and Jackson. The code boils down to something like this:

@GET
@Path("/data")
@ManagedAsync
public void getData(@Suspended final AsyncResponse response) {
    Stream<MyDataObject> dataStream = myDataBase.getData();

    // How to stream the data to the client 
    // instead of collecting it in a List first?
    List<MyDataObject> data = dataStream.collect(Collectors.toList());

    response.resume(Response.ok(data).build());
}

Instead of collecting the data in a List before sending it to the client, we would like to stream the results to the client since the amount of objects can be huge. From searches on google and stackoverflow I see I probably need StreamingOutput but I don't know how to connect the Stream via Jackson to Jersey.

In short: How to stream the results from the database directly to the client?

Jos de Jong
  • 6,602
  • 3
  • 38
  • 58
  • Just thinking out loud: I might try to write my own [MessageBodyWriter](https://jersey.java.net/documentation/latest/message-body-workers.html) for Stream that just forEachs the stream and [write](https://fasterxml.github.io/jackson-databind/javadoc/2.2.0/com/fasterxml/jackson/databind/ObjectMapper.html#writeValue(java.io.OutputStream,%20java.lang.Object)) each value with an ObjectMapper. There are solutions to Streams and Jackson that involve writing your own serializer. That would get you the output, but I'm not sure how that would work for the streaming requirement – Paul Samsotha Mar 21 '17 at 14:41
  • You don't have to use a MessageBodyWriter. You could just using a StreamingOutput. There's really nothing special about the StreamingOutput. It's just like writing to the OutputStream of the MessageBodyWriter. – Paul Samsotha Mar 21 '17 at 14:46
  • Actually, I don't think the ObjectMapper will work in this case. You will need to use Jackson Streaming API. You will need to get a JsonGenerator so you can write the start array and end array. See [the serialize function in this post](http://stackoverflow.com/a/34768093/2587435). Of course you will need to create your own generator though if you aren't using the custom serializer. – Paul Samsotha Mar 21 '17 at 15:41
  • Thanks @peeskillet I think you're right. Guess I will have to implement this plumbing myself to connect Stream to StreamingOutput. I'm really surprised that it looks like I'm the first one trying to do something like this, it sounds like a common use case to me: I can find zero libraries and hardly any tutorials or examples in this regard :(. Here is one tutorial though that I think will be useful: https://zackehh.com/collecting-a-java-8-stream-into-a-jackson-arraynode/. – Jos de Jong Mar 22 '17 at 07:33

0 Answers0