NodeJS: What's the difference between a Duplex stream and a Transform stream?

Question

The Stream docs state that Duplex Streams "are streams that implement both the Readable and Writable interfaces" and Transform Streams "are Duplex streams where the output is in some way computed from the input." Unfortunately, the docs do not describe what Transform streams provide above and beyond Duplex streams.

Are there any differences between the two? When would you use one over the other?

score 103 · Accepted Answer · edited Oct 16 '13 at 13:56

103

A Duplex stream can be thought of a readable stream with a writable stream. Both are independent and each have separate internal buffer. The reads and writes events happen independently.

                             Duplex Stream
                          ------------------|
                    Read  <-----               External Source
            You           ------------------|   
                    Write ----->               External Sink
                          ------------------|
            You don't get what you write. It is sent to another source.

A Transform stream is a duplex where the read and write takes place in a causal way. The end points of the duplex stream are linked via some transform. Read requires write to have occurred.

                                 Transform Stream
                           --------------|--------------
            You     Write  ---->                   ---->  Read  You
                           --------------|--------------
            You write something, it is transformed, then you read something.

edited Oct 16 '13 at 13:56

hopper

13,060
7
49
53

answered Aug 20 '13 at 15:33

user568109

47,225
17
99
123

1

I think I understand! Transform Streams must implement the `_transform` method, which has the same signature as a Writeable Stream's `_write` method, so this implies that what's read depends on what's written. One implication of this is that you can't specify the number of bytes to be read at a time like with `_read`, it's simply the number of bytes you wrote. On the other hand, with Duplex streams, you *could* theoretically make a `_read` which depends on `_write`, or even have the `_write` which depend on `_read`, but no read/write linking is implied by the Duplex Stream class itself. – brainkim Aug 21 '13 at 15:27
15

+1 for the ascii diagram, they should be used in the node.js docs that would help convey the concepts much better – David Chase Nov 24 '15 at 19:41
upvoted so by your definition a single transform stream should only generate a single transformed output, so lets say I have 1m ohlc data coming live through a stream and I want to generate 5m 15m etc from it, I would only do 1 conversion per instance of transform stream or would it be considered good practice to push all the timeframes from a single input? – PirateApp Nov 24 '18 at 16:42
Aren't the two types of streams implemented in the same way though, and just presented to us developers with different methods? (write and read for duplex, transform for transform streams) – Marcos Pereira Jun 28 '19 at 17:07

score 10 · Answer 2 · edited Oct 25 '13 at 16:45

10

The difference is just syntactic sugar. Transform streams are full duplex streams but rather than implementing _write and _read methods, you are asked to implement just the _transform method. You can read more about streams on the excellent substack's streams guide or from Isaacs's readable-stream repo.

edited Oct 25 '13 at 16:45

Jay

18,959
11
53
72

answered Aug 20 '13 at 12:56

danielepolencic

4,905
1
26
25

score 4 · Answer 3 · edited Jun 20 '20 at 09:12

According to the docs:

Duplex - streams that are both Readable and Writable (for example, net.Socket).

Transform - Duplex streams that can modify or transform the data as it is written and read (for example, zlib.createDeflate()).

So to put it simply:

A duplex stream can be read from and written to, but there's not necessarily any connection between the input and output.
A transform stream can also be read from and written to, but the output will always be the result of a transformation on its input. Transform streams are Duplex streams where the output is in some way related to the input.

score 2 · Answer 4 · answered Aug 20 '13 at 13:17

If you read the API for stream implementors section of the docs, they state that some possible use cases for Duplex and Transform streams are "Reading and writing" and "Operate on written data, then read the result" respectively.

Simply put the Transform stream lets you implement the _transform method that takes some input and returns the output after some operation has been done on the data, and can be used for things like compressing or hashing, whereas the Duplex stream can be used for things like a TCP socket connection where you just send and receive data.

NodeJS: What's the difference between a Duplex stream and a Transform stream?

4 Answers4

Linked