1

I'm working within the Django admin, and I've changed the form submissions to create a FormData object and then upload via ajax with jquery.

This works great when there aren't too many large files in the upload, but over a certain limit, I get hit with an Nginx (and possibly also uWSGI) errors that tell me my request object is too large. The obvious solution is to set this attribute higher (we've been told to expect regular uploads of 500MB - 2GB) but the team is wary about this causing issues on the server side, and I honestly have no idea whether or not sending a 1GB HttpRequest is a bad idea.

If there's no downside to upping the Nginx and uWSGI attributes, we'll go ahead and do that. If there are, I think my question becomes more about Django/javascript, and wondering what the best way of chunking this upload would be (by best, I hopefully mean having to rewrite the least amount of application code :) )

Thanks for the help! Robert

Robert Townley
  • 3,414
  • 3
  • 28
  • 54

1 Answers1

2

Try utilizing Blob interface to upload File objects , Blob.slice() to divide upload into parts if initial size of Blob exceeds set byte length

var blob = new Blob(["0123456789"]);
var len = blob.size;
var arr = [];
// if file size exceeds limit ,
// `.slice()` file into two or more parts
if (len > 5) {
  arr.push(blob.slice(0, 5), blob.slice(5));
};

arr.forEach(function(b, index) {
  setTimeout(function() {
    var reader = new FileReader();
    reader.onload = function(e) {
      // do stuff
      // e.g., send `e.target.result` , half of original file object to server
      // spaced two seconds apart
      // reassemble "slices" of `Blob` server side
      document.body.appendChild(
        document.createTextNode(
          e.target.result + "\n\n"
        )
      )
    }
    reader.readAsText(b);
  }, index * 2000);
});
guest271314
  • 1
  • 15
  • 104
  • 177
  • Thank you that's very helpful (also I hadn't heard of the FileReader object. Nifty new tool :) ) Not sure if you have any input on the Django side of things, but can you think of an example of how I might receive each of these chunks, piece-by-piece on the server side? – Robert Townley Jul 05 '15 at 17:33
  • _"can you think of an example of how I might receive each of these chunks, piece-by-piece on the server side?"_ Initially post total slices of `Blob` to be sent for that `Blob` , process each slice , save into an array of slices for that `Blob` ? See also http://stackoverflow.com/questions/3024892/ , http://stackoverflow.com/questions/492549/ , http://stackoverflow.com/questions/24306335/ , http://www.techcubetalk.com/tutorial-on-how-to-store-images-in-mysql-blob-field/ , http://www.phpeveryday.com/articles/PDO-Working-With-BLOBs-P554.html – guest271314 Jul 05 '15 at 17:58
  • @RobertTownley See also "Linked" , "Related" at right vertical column of page – guest271314 Jul 05 '15 at 18:03