Questions tagged [node.js-stream]

A stream is an abstract interface for working with streaming data in Node.js. The stream module provides a base API that makes it easy to build objects that implement the stream interface.

There are many stream objects provided by Node.js. For instance, a request to an HTTP server and process.stdout are both stream instances.

Streams can be readable, writable, or both. All streams are instances of EventEmitter.

The stream module can be accessed using:

const stream = require('stream');

There are four fundamental stream types within Node.js:

  • Readable - streams from which data can be read (for example fs.createReadStream()).
  • Writable - streams to which data can be written (for example fs.createWriteStream()).
  • Duplex - streams that are both Readable and Writable (for example net.Socket).
  • Transform - Duplex streams that can modify or transform the data as it is written and read (for example zlib.createDeflate()).
205 questions
101
votes
7 answers

Node.js Piping the same readable stream into multiple (writable) targets

I need to run two commands in series that need to read data from the same stream. After piping a stream into another the buffer is emptied so i can't read data from that stream again so this doesn't work: var spawn =…
Maroshii
  • 3,937
  • 4
  • 23
  • 29
77
votes
3 answers

How to implement a writable stream

I want to pipe data from an amazon kinesis stream to a an s3 log or a bunyan log. The sample works with a file write stream or stdout. How would I implmeny my own writable stream? //this works var file =…
MonkeyBonkey
  • 46,433
  • 78
  • 254
  • 460
70
votes
3 answers

How to wrap a buffer as a stream2 Readable stream?

How can I transform a node.js buffer into a Readable stream following using the stream2 interface ? I already found this answer and the stream-buffers module but this module is based on the stream1 interface.
Jerome WAGNER
  • 21,986
  • 8
  • 62
  • 77
35
votes
3 answers

How to pipe one readable stream into two writable streams at once in Node.js?

The goal is to: Create a file read stream. Pipe it to gzip (zlib.createGzip()) Then pipe the read stream of zlib output to: 1) HTTP response object 2) and writable file stream to save the gzipped output. Now I can do down to 3.1: var gzip =…
esengineer
  • 9,514
  • 7
  • 45
  • 69
26
votes
6 answers

How to mock streams in NodeJS

I'm attempting to unit test one of my node-js modules which deals heavily in streams. I'm trying to mock a stream (that I will write to), as within my module I have ".on('data/end)" listeners that I would like to trigger. Essentially I want to be…
flacnut
  • 1,030
  • 4
  • 11
  • 21
25
votes
1 answer

Pipe to stdout and writeable stream

I'm piping a file through a duplex string (courtesy of through) and I'm having trouble printing information to stdout and writing to the file. One or the other works just fine. var fs = require('fs'); var path = require('path'); var through =…
Nick Tomlin
  • 28,402
  • 11
  • 61
  • 90
22
votes
4 answers

What are the purposes of vinyl-buffer and gulp-streamify in gulp?

As the documentation says, they both deal with transforming non-stream plugins to stream. What I try to understand is, if I can use the .pipe() method on something, doesn't it mean it's a stream? If so, what do I convert to what…
Asaf Katz
  • 4,608
  • 4
  • 37
  • 42
20
votes
2 answers

Creating a Node.js stream from two piped streams

I'd like to combine two Node.js streams into one by piping them, if possible. I'm using Transform streams. In other words, I'd like my library to return myStream for people to use. For example they could…
Nicolas Hery
  • 1,068
  • 1
  • 10
  • 8
19
votes
3 answers

What is the difference between local and global module in Node.js? When to use local and global module?

We can access local module using require function but cannot access global module through it. I read somewhere that to use global module we need to make it local then import it through require function. So if we cannot access global module…
Badal
  • 376
  • 4
  • 11
19
votes
1 answer

Multiple Consumption of single stream

I want to know if its possible that multiple functions can consume single stream in node.js. If yes How can this done? Is it possible to pipe to multiple destinations? I want to use the stream in two different functions which are parallel. I am…
Saransh Mohapatra
  • 9,430
  • 10
  • 39
  • 50
15
votes
1 answer

Proper way to unpipe a streams2 pipeline and empty it (not just flush)

Premise I'm trying to find the correct way to prematurely terminate a series of piped streams (pipeline) in Node.js: sometimes I want to gracefully abort the stream before it has finished. Specifically I'm dealing with mostly objectMode: true and…
zamnuts
  • 9,492
  • 3
  • 39
  • 46
10
votes
1 answer

Stream file to html video player as it's being downloaded in Electron using fs

I'm currently trying to use the HTML video player to stream a file from the file system in Electron. I would like to start streaming as the file is downloading. I'm not sure if my current plan will work (or if this is even possible). The…
sss
  • 618
  • 1
  • 8
  • 20
9
votes
2 answers

Node.js: splitting stream content for n-parts

I'm trying to understand node streams and their life-cycle. So, I want to split the content of a stream for n-parts. The code below is just to explain my intentions and to show that I already try something by myself. I omitted some details I have a…
kharandziuk
  • 12,020
  • 17
  • 63
  • 121
9
votes
2 answers

Node.js "write after end" error

Starting off with a basic node app, and I can't figure out how to beat this "write after end" error, even with the callbacks suggested on several sites. index.js: var server = require("./server"); var router = require("./router"); var…
francojposa
  • 185
  • 1
  • 2
  • 14
9
votes
3 answers

Node.js Streams Readable to Transform

I have been trying to use a readable and a transform stream to process a very large file. The problem that I seem to come across is that if I don't put a writable stream at the end, the program seems to terminate before the result gets…
ace040686
  • 153
  • 1
  • 9
1
2 3
13 14