1

I am trying to achieve following error handling:

  1. Say we have a readable stream.
  2. We pipe it into a transform stream.
  3. Somehow the transform stream emits an error.
  4. We would like to recover the readable stream (and all of its data), and re-pipe it into another transform stream.

Step 4 appears to be difficult: I can listen to unpipe event on the target stream (transform stream) and retrieve a reference to the source stream (readable stream), but at least some chunks of its data have been lost.

Can we do this without creating a custom transform stream?

A real-world example is deflate content encoding, where in some cases, you need zlib.createInflateRaw() instead of zlib.createInflate(), but we can't decide which one would be the correct choice before looking at the response body buffers.

Community
  • 1
  • 1
bitinn
  • 9,188
  • 10
  • 38
  • 64
  • Don't most sites support gzip these days anyway? Just send `Accept-Encoding: gzip` in your request and you don't have to worry about any deflate issues. – mscdex Aug 03 '16 at 09:03
  • @mscdex you are right but for a http library we can't ignore those old and crazy deflate-only servers, I ended up doing this, seems like the best I can do at the moment. https://github.com/bitinn/node-fetch/blob/master/index.js#L209-L222 – bitinn Aug 03 '16 at 10:38

1 Answers1

2

You do not need to introduce a stream in the middle just to read the first byte. For example:

(function readChunk() {
  var chunk = res.read();
  if (!chunk)
    return res.once('readable', readChunk);

  var inflate;
  if ((chunk[0] & 0x0F) === 0x08)
    inflate = zlib.createInflate();
  else
    inflate = zlib.createInflateRaw();

  // Put chunk back into stream
  res.unshift(chunk);

  // Prepare the decompression
  res.pipe(inflate);

  output = new Response(inflate, response_options);
  resolve(output);
})();

Also, var body = res.pipe(new stream.PassThrough()); is unnecessary, you can just use res where appropriate.

mscdex
  • 104,356
  • 15
  • 192
  • 153