NodeJS race condition?

Question

I currently have a piece of code that downloads a file onto the local filesystem. It's fairly straightforward:

var downloadFile = function(request, file_name, cb) {
  var final_path = file_name + '.pdf';
  var file = fs.createWriteStream(final_path);

  // Save temporary file to disk
  var save_file = request.pipe(file);

  save_file.on('finish', function () {
    cb('success', final_path);
  });
}

However, on occasion, I would encounter an error (later on in the code) that suggests that the file had not been fully downloaded. When I open the file manually myself, it appears fine. Additionally, the bug seems to happen more frequently when we have a fairly large amount of users simultaneously on the site.

All of this clearly suggests to me a possible race condition, but I'm not exactly sure where or how this could be happening. My understanding of node is that it's single threaded with non-blocking I/O, which would suggest any possible culprits be in the I/O parts: the createWriteStream and subsequent pipe (possibly the 'finish' callback handler). Perhaps they are being overwritten somehow or called out-of-order. But how? The finish handler seems like a standard promise/callback. Any insights would be greatly appreciated!

-Edward

It could be that the finish callback has not been executed yet, at that point later in the code, when you try to access the file (if that is what you are doing). Could you post the code that gives the error too? — Tom Erik Støwer, Sep 16 '14 at 06:03
the "cb('success', final_path)" function contains all the future code. Nothing is executed until the 'finish' handler is reached. But could one user evoke the finish handler for another user's save_file? (Since this is I/O, which could potentially cause conflicts?) — Edward Sun, Sep 16 '14 at 14:20
@TeemuIkonen it appears 'finish' is the event to listen for according to node's docs for a WriteStream: http://nodejs.org/api/stream.html#stream_event_finish -- but am I overlooking something? — Edward Sun, Sep 16 '14 at 15:01

score -1 · Accepted Answer · answered Sep 16 '14 at 10:02

-1

Try this:

var downloadFile = function(request, file_name, cb) {
  var final_path = file_name + '.pdf';
  var file = fs.createWriteStream(final_path);

  // Save temporary file to disk
  request.pipe(file);

  request.on('end', function () {
    cb('success', final_path);
  });
}

answered Sep 16 '14 at 10:02

Vickrant

1,233
2
9
15

I read on various Stackoverflow posts (http://stackoverflow.com/questions/11447872/callback-to-handle-completion-of-pipe and http://stackoverflow.com/questions/15462086/node-js-v0-10-stream2-api-pipe-end-event) that seem to be confused as to whether to use 'end', 'finish' or 'close'. Looking at node's docs (http://nodejs.org/api/stream.html#stream_event_finish), it looks like finish is currently the way to go. – Edward Sun Sep 16 '14 at 14:39
Here we are referring request so in this case we will be using 'end'. – Vickrant Sep 17 '14 at 10:16

NodeJS race condition?

1 Answers1