7

I used busboy module to parse multipart request with below coffeeScript code. The problem is, sometimes, on 'data' handler called several times for the request including one file. That means I need to sum to each size to figure the whole size. Besides the file object in the on 'file' handler seems not including size information.

How to get the whole size without calculating each part?

Thanks in advance-

busboy.on 'file', (fieldname, file, filename, encoding, mimetype) ->
  filename = "#{Meteor.uuid()}.jpg"
  dir = "#{HomeDir()}/data/profile"
  saveTo = path.join dir, filename
  file.pipe fs.createWriteStream saveTo
   files.push
     filename: filename
     path: saveTo
     fileSize: data.length
  file.on 'data', (data) ->
    # this data handler called several times 
    files.push
      filename: filename
      path: saveTo
      fileSize: data.length    
  file.on 'end', ->
    console.log 'file finished'
trex005
  • 5,015
  • 4
  • 28
  • 41
kakadais
  • 441
  • 1
  • 4
  • 17

2 Answers2

8

Since you're already piping the stream to a file, you need to use something like stream-meter:

var meter = require('stream-meter');
...
busboy.on('file', function(fieldname, file, filename, encoding, mimetype) {
  ...
  var m = meter();
  file.pipe(m).pipe(fs.createWriteStream(saveTo)).on('finish', function() {
    files.push({
      filename : filename,
      path     : saveTo,
      fileSize : m.bytes,
    });
  });
});
robertklep
  • 198,204
  • 35
  • 394
  • 381
  • My `m.bytes` comes in as 0 even though I know the file isn't 0 bytes in size – user3871 Feb 10 '17 at 22:24
  • @Growler try checking for `error` events, it might be that an error occurs that prevents the file from being written (I _think_ that it will still trigger the `finish` event after such a thing) – robertklep Feb 11 '17 at 07:53
5

This is a very late response. I hope it'll help.

request.headers['content-length'] will give you the entire file size. You don't need busboy for that. It is a part of the request object and can be accessed as follows:

http.createServer(function(request, response) {

      /* request handling */

      console.log("File size:" + request.headers['content-length'] / 1024 + "KB");

      /* busboy code to parse file */

    }

An example which uses the header size info to track file transfer progress can be found at: NodeJS - file upload with progress bar using Core NodeJS and the original Node solution

EDIT:

As noted by @Matthias Tylkowski, content-length is not same as file size.

My own code has also changed since then. I noticed that in the file up-loader that I've implemented using NODEJS, I've read file size directly using HTML5 File API and passed it to the nodejs server.

jim1427
  • 140
  • 2
  • 9
  • 3
    The content-length is not the file size, as there is some overhead for marking the boundaries – Matthias Tylkowski Jul 25 '19 at 13:20
  • @MatthiasTylkowski Thanks for noting. You're right. Content length is different from file size. Will edit the answer. However I think content-length will serve as an approximate value for calculating overall file transfer progress. – jim1427 Jul 26 '19 at 10:11
  • 3
    If the request contains several files - 'content-length' will show a sum of all files size + tiny overhead. If the request is a form data - it might contain non-files fields. That also impacts content-length. – tlumko Nov 06 '19 at 11:16
  • This value cannot be relied upon as an attacker can simple change the value of this header – DollarAkshay Feb 24 '21 at 07:00
  • What if you are passing multiple files in the request? – MrOli3000 Oct 16 '22 at 06:37
  • 1
    @MrOli3000 As mentioned in my edit, I'm reading file info using HTML5 API and passing filesize, name and other info in JSON format to the server. It doesn't matter if there is a single file or more files. For details on getting info of multiple files using HTML5 API: https://developer.mozilla.org/en-US/docs/Web/API/File_API/Using_files_from_web_applications#getting_information_about_selected_files – jim1427 Oct 19 '22 at 06:06