Node stream buffers in console.log vs process.stdout.write

Question

Using NodeJS v5.6 I created a file called read-stream.js:

const
    fs = require('fs'),
    stream = fs.createReadStream(process.argv[2]);

stream.on('data', function(chunk) {
    process.stdout.write(chunk);
});

stream.on('error', function(err) {
    process.stderr.write("ERROR: " + err.message + "\n");
});

and a data file in plain text called target.txt:

hello world
this is the second line

If I do node read-stream.js target.txt the contents of target.txt are printed normally on my console and all is well.

However if I switch process.stdout.write(chunk); with console.log(chunk); then the result I get is this:

<Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0a 74 68 69 73 20 69 73 20 74 68 65 20 73 65 63 6f 6e 64 20 6c 69 6e 65 0a>

I recently found out that by doing console.log(chunk.toString()); the contents of my file are once again printed normally.

As per this question, console.log is supposed to use process.stdout.write with the addition of a \n character. But what exactly is happening with encoding/decodings here?

Thanks in advance.

score 7 · Answer 1 · edited Jan 19 '19 at 01:32

7

process.stdout is a stream and its write() function only accepts strings and buffers. chunk is a Buffer object, process.stdout.write writes the bytes of data directly in your console so they appear as strings. console.log builds a string representation of the Buffer object before outputting it, hence the <Buffer at the beginning to indicate the object's type and following are the bytes of this buffer.

On a side note, process.stdout being a stream, you can pipe to it directly instead of reading every chunk:

stream.pipe(process.stdout);

edited Jan 19 '19 at 01:32

Alexis Wilke

19,179
10
84
156

answered Feb 17 '16 at 11:42

Shanoor

13,344
2
29
40

1

Using `pipe` is brilliant! – jiajianrong Jul 04 '17 at 13:45

score 2 · Answer 2 · answered Feb 17 '16 at 14:52

I believe I found out what's happening:

The implementation of console.log in NodeJS is this:

Console.prototype.log = function() {
    this._stdout.write(util.format.apply(this, arguments) + '\n');
};

However the util.format function of lib/util.js in NodeJS uses the inspect method on any input object which in turn: Returns a string representation of object, which is useful for debugging.

Thus what's happening here is that due to util.format "object casting", anytime that we pass an object to console.log, that particular object is first turned into a string representation and then is passed to process.stdout.write as a string and finally gets written to the terminal.

So, when we directly use process.stdout.write with buffer objects, util.format is completely skipped and each byte is directly written to terminal as process.stdout.write is designed to handle them directly.

Node stream buffers in console.log vs process.stdout.write

2 Answers2

Linked