0

I'm creating a ByteArrayOutputStream using ZIO Streams i.e.:

lazy val byteArrayOutputStream = new ByteArrayOutputStream()
val sink = ZSink.fromOutputStream(byteArrayOutputStream).contramapChunks[String](_.flatMap(_.getBytes)
val data = ZStream.unwrap(callToFunction).run(sink)

This works fine - now I need to stream this data back to the client using akka http. I can do this:

val arr = byteArrayOutputStream.toByteArray
complete(HttpEntity(ContentTypes.`application/octet-stream`, arr)

which works but of course the toByteArray brings the outputstream into memory i.e. I don't stream the data. I'm missing something obvious - is there an easy way to do this?

user1128482
  • 185
  • 1
  • 1
  • 11

1 Answers1

0

You can convert output stream to Akka Stream Source:

val byteArrayOutputStream = new ByteArrayOutputStream()
val source = StreamConverters.asOutputStream().mapMaterializedValue(_ => byteArrayOutputStream)

and then simply create a chunked HTTP entity:

HttpResponse(entity = HttpEntity.Chunked.fromData(ContentTypes.`application/octet-stream`, source))

More about chunked transfer: https://datatracker.ietf.org/doc/html/rfc7230#section-4.1

For ZIO, you could probably use something like this:

val zSource = ZStream.fromOutputStreamWriter(os => byteArrayOutputStream.writeTo(os))

However, you need to find a way to convert ZStream to Akka Stream Source.

Branislav Lazic
  • 14,388
  • 8
  • 60
  • 85
  • https://github.com/zio/interop-reactive-streams might help with converting between ZStream and Akka Stream – PJ Fanning Nov 30 '21 at 00:35
  • Thanks @Branislav for your answer. However when I implement and run it the request never returns. I notice from the API that it's inherently blocking so I created a custom dispatcher - unfortunately again no joy. Anything that I'm missing? – user1128482 Nov 30 '21 at 09:30
  • @PJ Fanning - yes I looked at that, however the examples are not that great, and it uses code from reactive.org which has not been updated in years the website is no longer available. Not something I'll easily be able to justify using in a corporate environment – user1128482 Nov 30 '21 at 09:33