1

I have a large list of objects(possibly 1k-2k stored in db) on the server side that needs to be sent to multiple clients using Akka Http on client's request. The simple solution is I can retrieve it from DB on the server side, make Bytes of a list of Objects and then send bytes to a client and later convert that byte to object.

But the issue is it can impact memory performance on the client as well as a server as the list is very large.

So is there any best possible way to achieve this other than converting the whole list of an object into bytes?

Ramón J Romero y Vigil
  • 17,373
  • 7
  • 77
  • 125
rns
  • 1,047
  • 10
  • 25

1 Answers1

1

The answer to your question involves several steps for both server side and client side. Note: all of the code examples below are in scala rather than java for terseness. On the server side you need to implement:

1. Streaming the Data from a DB

There is another stack question/answer that explains how to get rows out of a database using streaming methods. The end result is that you have a source of rows:

type DataObject = ???

val source : Source[DataObject,_] = ??? // see linked answer

2. Converting DataObject to Bytes

This part can only be answered knowing the specifics of your data and it sounds like you already have this part implemented:

val converter : DataObject => ByteString

3. Attaching to a REST Endpoint

You can use akka-http to setup a REST endpoint that clients can query to get the data. The Source of your ByteString values will be the Source feeding the stream entity for your HttpResponse

Setup a Route that responds to GET requests on a particular path:

val route : Route = 
  get {
    path("/queryDB") {
      complete(HttpResponse(entity = HttpEntity(ContentTypes.`application/octet-stream`, source.map(converter))))
    }
  }

And use the Route to launch your service:

Http().bindAndHandle(route, "localhost", 8080)

On the client side there are similar mechanisms for reading data from an HttpResponse.

Ramón J Romero y Vigil
  • 17,373
  • 7
  • 77
  • 125
  • Thanks @Ramon J. Do I need to convert source to bytes and then send to client? – rns Nov 30 '18 at 07:02
  • @rns You are welcome. I don't fully understand your question. The `HttpEntity` takes in a source *of* bytes (actually bytestrings but basically bytes). Therefore, you don't convert the actual source to a bytestring. – Ramón J Romero y Vigil Nov 30 '18 at 11:50
  • @Thanks Ramon It worked by Slick, Streams and akka-http.| I have one more question, Is it possible to use streaming in existing Http application eg Spring or Camel and integrate Streams in it? – rns Dec 10 '18 at 06:11
  • 1
    @rns I don't know the answer to that question as I have never used Spring or Camel. I recommend a new stack question... – Ramón J Romero y Vigil Dec 10 '18 at 11:44