11

What is the basic difference between these two operations ?

someReadStream.pipe(fs.createWriteStream('foo.png'));

vs

someReadStream.on('data', function(chunk) { blob += chunk } );
someReadStream.on('end', function() { fs.writeFile('foo.png', blob) });

When using request library for scraping, I can save pics (png, bmp) etc.. only with the former method and with the latter one there is same gibbersh (binary) data but image doesn't render.

How are they different ?

SomeGuyOnAComputer
  • 5,414
  • 6
  • 40
  • 72
Randy
  • 153
  • 1
  • 2
  • 7

1 Answers1

9

When you are working with streams in node.js you should prefer to pipe them.

According to Node.js’s stream-event docs, data events emit either buffers (by default) or strings (if encoding was set).

When you are working with text streams you can use data events to concatenate chunks of string data together. Then you'll be able to work with your data as one string.

But when working with binary data it's not so simple, because you'll receive buffers. To concatenate buffers you use special methods like Buffer.concat. It's possible to use a similar approach for binary streams:

var buffers = [];
readstrm.on('data', function(chunk) {
    buffers.push(chunk);
});
readstrm.on('end', function() {
    fs.writeFile('foo.png', Buffer.concat(buffers));
});

You can notice when something goes wrong by checking the output file's size.

jschoi
  • 1,884
  • 1
  • 12
  • 26
Leonid Beschastny
  • 50,364
  • 10
  • 118
  • 122
  • 1
    thanks that was eyeopening for me but i cant achieve it the 'hard way' too! . var buffers = []; readstrm.on('data', function(chunk) { var blob = new Buffer(chunk.length); buffers.push(blob); }); readstrm.on('end', function() { var finalblob = Buffer.concat(buffers); fs.writeFile('img.bmp', finalblob); }); – Randy Jan 05 '13 at 10:59
  • I did it once and it worked... There was no `concat` in node 0.6.x though, so I used some node module from github. – Leonid Beschastny Jan 05 '13 at 18:29
  • I updated my answer, but my code looks pretty much like yours. – Leonid Beschastny Jan 05 '13 at 18:56