Read all text from stdin to a string

Question

I'm writing a program in Node.js that (in some situations) wants to act as a simple filter: read everything from stdin (up to end of file), do some processing, write the result to stdout.

How do you do the 'read everything from stdin' part? The closest solutions I've found so far, seem to work either for one line at a time from the console, or else only work when stdin is a file not a pipe.

Answer 1

My boiler-plate for this one is a lot like the solution described in a comment above -- offering it at the top level because it's very much the simplest way to do this and it shouldn't be only in a comment.

var fs = require('fs');
var data = fs.readFileSync(0, 'utf-8');
// Data now points to a buffer containing the file's contents

Answer 2

I use the following in Node 11+

 async function read(stream) {
   const chunks = [];
   for await (const chunk of stream) chunks.push(chunk); 
   return Buffer.concat(chunks).toString('utf8');
 }

Usage:

const input = await read(process.stdin);

Answer 3

If you're on linux, there is no need for a 3rd party package for this. of course, consider your performance needs, but these two lines will work:

const fs = require("fs");
const data = fs.readFileSync("/dev/stdin", "utf-8");

Jan points out in the comments below that a more portable solution would be to use 0, as this is the POSIX standard. So, you may simple use:

const fs = require("fs");
const data = fs.readFileSync(0, "utf-8");

data is now a string with your data from stdin, interpreted as utf 8

Answer 4

get-stdin will do the trick.

---

A few notes reading between the lines in your question.

Since you tagged the question "synchronous" I'll just note that stdin is async-only in node.js. The above library is the simplest it gets. It will handle the entire input as either a string or a buffer.

If possible, writing your program in the streaming style is best, but some use cases are feasible for streaming (i.e. word count) and some are not (i.e. reverse the input).

Also the "one line at a time from the console" is an artifact of the terminal buffering your keystrokes. If you want some "I'm sorry I asked" level detail check out the amazing the TTY Demystified.

Answer 5

I haven't seen a solution here that is actually synchronous except for @Patrick Narkinsky's. But @Patrick Narkinsky's answer doesn't work on Windows. Seems to be a node.js bug. If you want to learn the details, feel free to go down this rabbit hole of github issues, but I gave up after an hour of reading.

1. https://github.com/aws/aws-cdk/issues/11314 The issue reported here
2. https://github.com/nodejs/node/issues/35997 A contributor from that library creates a node issue
3. https://github.com/libuv/libuv/pull/3053 A nodejs contributor submits a PR with a fix(?) (not yet merged)

I wasn't able to find a workaround there (I probably glossed over it), but I accidentally stumbled on a solution to the problem. It's not pretty, but it works. Since that link only shows how to log the progress, I had to modify it for my own needs:

import fs from 'fs';
const BUFSIZE = 256;
const buf = Buffer.alloc(BUFSIZE);
let bytesRead;
let stdin = '';

export function stdinToString(): string {
  do {
    // Loop as long as stdin input is available.
    bytesRead = 0;
    try {
      bytesRead = fs.readSync(process.stdin.fd, buf, 0, BUFSIZE, null);
    } catch (e) {
      if (e.code === 'EAGAIN') {
        // 'resource temporarily unavailable'
        // Happens on OS X 10.8.3 (not Windows 7!), if there's no
        // stdin input - typically when invoking a script without any
        // input (for interactive stdin input).
        // If you were to just continue, you'd create a tight loop.
        throw 'ERROR: interactive stdin input not supported.';
      } else if (e.code === 'EOF') {
        // Happens on Windows 7, but not OS X 10.8.3:
        // simply signals the end of *piped* stdin input.
        break;
      }
      throw e; // unexpected exception
    }
    if (bytesRead === 0) {
      // No more stdin input available.
      // OS X 10.8.3: regardless of input method, this is how the end
      //   of input is signaled.
      // Windows 7: this is how the end of input is signaled for
      //   *interactive* stdin input.
      break;
    }
    // Process the chunk read.
    stdin += buf.toString(undefined, 0, bytesRead);
  } while (bytesRead > 0);

  return stdin;
}

I've been programming for over a decade and this is the first time a do while made my code cleaner :) Without it, this function would hang if no stdin data exists -- one could argue that was a bug in the code of that link.

This answers the original question AND works on all operating systems.