2

I'm writing my own through stream in Node which takes in a text stream and outputs an object per line of text. This is what the end result should look like:

fs.createReadStream('foobar')
  .pipe(myCustomPlugin());

The implementation would use through2 and event-stream to make things easy:

var es = require('event-stream');
var through = require('through2');
module.exports = function myCustomPlugin() {
  var parse = through.obj(function(chunk, enc, callback) {
    this.push({description: chunk});
    callback();
  });
  return es.split().pipe(parse);
};

However, if I were to pull this apart essentially what I did was:

fs.createReadStream('foobar')
  .pipe(
    es.split()
      .pipe(parse)
  );

Which is incorrect. Is there a better way? Can I inherit es.split() instead of use it inside the implementation? Is there an easy way to implement splits on lines without event-stream or similar? Would a different pattern work better?

NOTE: I'm intentionally doing the chaining inside the function as the myCustomPlugin() is the API interface I'm attempting to expose.

Sukima
  • 9,965
  • 3
  • 46
  • 60
  • Not sure why downvoted. This is my question exactly and seems like a natural but not entirely straightforward pattern. –  Jul 16 '15 at 11:24

2 Answers2

1

Based on the link in the previously accepted answer that put me on the right googling track, here's a shorter version if you don't mind another module: stream-combiner (read the code to convince yourself of what's going on!)

var combiner = require('stream-combiner')
  , through = require('through2')
  , split = require('split2')

function MyCustomPlugin() {
  var parse = through(...)

  return combine( split(), parse )
}
0

I'm working on something similar. See this solution: Creating a Node.js stream from two piped streams

var outstream = through2().on('pipe', function(source) {
    source.unpipe(this);
    this.transformStream = source.pipe(stream1).pipe(stream2);
});
outstream.pipe = function(destination, options) {
    return this.transformStream.pipe(destination, options);
};
return outstream;
Community
  • 1
  • 1
Kenny Au
  • 16
  • 3
  • 1
    I think this is the way to go. However, the SO question/answer linked to didn't illuminate where `stream1` and `stream2` are defined. – Sukima Nov 17 '14 at 17:19
  • Yeah I think it is too. All of that code would be in your "myCustomPlugin()" function. You can replace "stream1" and "stream2" with "es.split()" and "parse" respectively. Your parse stream can be defined anywhere, really. (i.e. within the myCustomPlugin function, in an external file and referenced at the top, etc.) – Kenny Au Nov 17 '14 at 17:42