3

I am having an issue with detecting end of piped stream. steps:

  1. Read File
  2. Creating gzip
  3. Write File

Which is the right way to detect end of piped stream read and write.

var inp = fs.createReadStream(fileName);
var out = fs.createWriteStream(fileName + '.gz');
var gzip = zlib.createGzip();
inp.pipe(gzip).pipe(out);
out.on('finish', function () { ....

vs

var inp = fs.createReadStream(fileName);
var out = fs.createWriteStream(fileName + '.gz');
var gzip = zlib.createGzip();
var zipStream = inp.pipe(gzip).pipe(out);
zipStream.on('finish', function () { ....

reference Link

Community
  • 1
  • 1
Ratan Kumar
  • 1,640
  • 3
  • 25
  • 52

2 Answers2

8

As explained in the documentation https://nodejs.org/api/stream.html:

  • 'finish': is emitted after the stream.end() method has been called, and all data has been flushed to the underlying system.
  • 'end': is emitted when there is no more data to be consumed from the stream.

Events: 'finish' and 'end'

The 'finish' and 'end' events are from the stream.Writable and stream.Readable classes, respectively. The 'finish' event is emitted after stream.end() is called and all chunks have been processed by stream._transform(). The 'end' event is emitted after all data has been output, which occurs after the callback in transform._flush() has been called.

So, in your example, the order will be:

  • inp (readable stream) will emit the 'end'
  • gzip (transformer): will emit a 'end' and 'finish' (transformer is duplex stream)
  • out (writable stream): will finally emit a 'finish' when all data are written.
Dario
  • 3,905
  • 2
  • 13
  • 27
  • so, I should only be using out.on('finish', function () { .. to do someting when I expect writing has been finished. – Ratan Kumar Nov 29 '16 at 12:26
  • Your second option `zipStream = inp.pipe(gzip).pipe(out); zipStream.on('finish', function () { .... })` is correct. Because the last pipe, returns the last stream (the writable one). – Dario Nov 29 '16 at 12:30
  • there was a typo in first option, can you tell why out.on('finish', function () { .. will not work ? – Ratan Kumar Nov 29 '16 at 12:34
  • 1
    Is the same, will work both. Because out === inp.pipe(gzip).pipe(out) – Dario Nov 29 '16 at 12:42
-1
var inp = fs.createReadStream(fileName);
var out = fs.createWriteStream(fileName + '.gz');
inp.pipe(gzip).pipe(out);
inp.on('finish', function () { ....

This is the best since you are listening on the input, plus that's how I used it last.

Akinjide
  • 2,723
  • 22
  • 28