0

Using Dropzone.js and the Dropbox API, I'm looking to setup a drag-and-drop web interface that can accept files up to 10G. It works perfectly with files up to ~50M, but fails after that with a 413 Request Entity Too Large error. The sample file I've been trying is 1.5G. I've tried what I know to change for large file uploads, but still doesn't seem to be working. I'm hoping someone can point me in the right direction with a look at my PHP settings.

Here's a rendering of the associated phpinfo(): http://codepen.io/jboneca/pen/bNvEwV

Additional information:

  • No file is ever stored on our server. It is uploaded (successfully, in most cases) directly to Dropbox.
  • File size limitation has been disabled on Dropzone.
  • The following settings have been changed, to no avail.upload_max_filesize, post_max_size, max_execution_time, max_input_time, memory_limit. Perhaps they need to be still larger?
emsoff
  • 1,585
  • 2
  • 11
  • 16

1 Answers1

2

A few things to note:

  1. When you handle a file upload with php, even if your script does not end up saving the file but only to send it to the Dropbox API -- the way it works is php must write it to a temp location on the server, e.g. /tmp/tempfilename. Therefore your server must have enough space to store the file, even though it's just temporary. Your script then uploads the file via Dropbox API and PHP will automatically clean up/remove the file from the temp location.
  2. The 413 Request Entity Too Large error typically happens when the post_max_size is not large enough to handle your request, but in your case you might have other issues limiting it as well.
  3. In my experience from running command line php scripts handling large amounts of data, the php memory limit starts breaking around 1700M-2000M. Meaning, even after setting memory_limit to something much higher, php itself could not handle checking memory limit that large, and fails when memory usage exceeds something around 1800M. (maybe it uses a signed 32-bit int for memory limit? Not sure.)
  4. I have worked with the combo of Dropzone.js + php file uploads as well, and had no problems uploading files of several hundred MBs. I recommend you to try this: set upload_max_filesize to 1100M, set post_max_size to 1200M, and set memory_limit to 1700M. Now, first try files of 100mb, then 500mb, then 1gb. I believe it will work for all of these sizes. Now adjust each setting higher and see how close you can get to 1.5gb. That's probably the limit of the current php version and server that you have.
  5. Just FYI, max_execution_time is not your issue if your error was 413 Request Entity Too Large. (you're not even there yet) This is only the time it takes to execute all the lines in your php script. (If you were saving the file on server locally, this time would be in milliseconds even if you upload a 1GB file.) However, since you need to send this file to Dropbox API from your server, it is possible that your request will take too long and over the max_execution_time limit. I believe you will run into problems with this after you solve your current issue with the upload. To solve this, I recommend your http request handling php script to simply add the file to a queue somewhere, and have a separate consumer process that reads from the queue and uploads files to Dropbox. (also probably better UX for your users so they don't have to wait 2x as long)
byl83
  • 736
  • 5
  • 8