3

I'm studying ZIO Streams, using version 1.0.9 of the library zio-streams. I cannot find any reference that shows me the difference between a ZSink and a ZTransducer.

What is the difference?

Yuval Itzchakov
  • 146,575
  • 32
  • 257
  • 321
riccardo.cardin
  • 7,971
  • 5
  • 57
  • 106

1 Answers1

3

A ZSink is a used to consume elements from the ZStream. In turn a sink will cause the materialization of the entire stream to run through and produce a value.

A ZTransducer is a stateful function which processes chunks of data in the stream, and in contrast to a ZSink it's processing never ends.

If we look at a trivial example:

import zio.stream.{ ZSink, ZStream }
import zio.stream.ZTransducer.{ splitLines, utf8Decode }

import java.nio.file.Paths

ZStream
  .fromFile(Paths.get("/foo"), chunkSize = 1024 * 4)
  .transduce(utf8Decode >>> splitLines)
  .run(ZSink.foreach(zio.console.putStr))

We can see that the new ZStream encoding is chunk based, and using transduce with utf8Decode which decodes a chunk of bytes to String and then splits them by line. Finally, we use the .run method and supply a ZSink which emits data to the console, element by element.

More on the new design of ZStream can be found here.

Yuval Itzchakov
  • 146,575
  • 32
  • 257
  • 321
  • Many thanks. So, if we have to transform values of a stream, but we want to maintain their cardinality, we use `map` (or its variants). If we want to transform both the values and the cardinality of chunks, we use a `ZTransducer`. Finally, if we want to consume the values of a stream, we use a `ZSink`. Have I understood right? – riccardo.cardin Jul 12 '21 at 07:20
  • 1
    @riccardo.cardin The question of cardinality of an unbounded stream is a difficult one. In that sense, AFAIU, there shouldn't be a change to cardinality even if your stream is bounded. A `ZTransducer` will provide you with the chance to operate on chunks of data instead of on an element by element basis. – Yuval Itzchakov Jul 12 '21 at 16:19
  • Many thanks. The current version of ZStreams uses _implicit chunking_; you're right. The only way to work explicitly with chunks is to use a `ZTransducer`. Maybe, you could add this last consideration to your answer :) – riccardo.cardin Jul 12 '21 at 19:57