7

I started learning NodeJS, and streams seems to be something that people use a lot. In most of the documentation that I had read there are mentions of the "back-pressure problem" that occurs when you are processing big sized files but I haven't found a clear explanation of what exactly this problem is. Also I have read that using pipes can help with this problem, but how exactly do pipes fix the back-pressure problem?

Thanks for any explanation in advance.

uzluisf
  • 2,586
  • 1
  • 9
  • 27
David Florez
  • 1,460
  • 2
  • 13
  • 26

1 Answers1

6

Backpressure is when you write to a stream faster than the other process can handle/consume Using Pipes, you can control the flow, pause and resume the stream; Here is an example implementing backpressure in nodejs

var http = require('http'),
    fs = require('fs');

var server = http.createServer(function(request, response) {
  var file = fs.createWriteStream('upload.jpg'),
      fileBytes = request.headers['content-length'],
      uploadedBytes = 0;

  request.on('data', function(chunk) {
    uploadedBytes += chunk.length;
    var progress = (uploadedBytes / fileBytes) * 100;
    response.write('progress: ' + parseInt(progress, 10) + '%\n');

    var bufferOK = file.write(chunk);

    if (!bufferOK) {
      request.pause();
    }
  });

  file.on('drain', function() {
    request.resume();
  });

  request.on('end', function() {
    response.end('upload complete\n');
  });

});

server.listen(8080);

Ben foster solution - Souce: https://gist.github.com/benfoster/9543337

Wagner Silveira
  • 96
  • 1
  • 5
  • how does client must react on pausing channel? can it continue throwing away requests assuming underlying transport system controls everything (actually not cause somewhere is limited buffer/queue)? – Simon Logic Jan 29 '21 at 14:19