2

I have a web service that accepts really huge files. Usually in the range of 10 - 15 GB (not MB). However upload using a browser is only possible using Chrome on Linux. All 3 major browsers have different flaws trying to upload such a file:

  • Internet Explorer stops after exactly 4GB.
  • Firefox does not start at all.
  • Chrome (on Windows) transfers the whole file but fails to send the closing bondary (send 0xff instead).

Now we are searching for a way to get uploads to work. Preferably using HTML/JS only, but I see no way to make that happen. Second try would be flash, but FileReference seems to break for files > 4GB. Last way would be Java but that is not what we are looking for in the browser client.

Note that this is about the client. I know that the server side code works, as I can upload a 12GB file using standard HTML-Upload with Chrome on Linux. It is the only browser/os combination that works so far, but therefor I am sure, the server coode is fine.

Does anyone know any way to get huge file uploads to work?

Regards, Steffen

Steffen Heil
  • 4,286
  • 3
  • 32
  • 35
  • this might help: "uploading a file in chunks using html5" http://stackoverflow.com/questions/7853467/uploading-a-file-in-chunks-using-html5 – ansiart Sep 17 '12 at 22:20
  • 1
    uploading 4gb at once is not ok, the temporary chunks are saved to ram memory so imagine what if 10 users upload 4gb file size at the same time. Strongly suggest to use chunks! – Adrian Pirvulescu Sep 18 '12 at 12:53
  • I am not sure, what you mean with chunks. I know about chunk-encoding, which is something out of control for javascript code. I have also read the answer linked in the first comment - but that would only work in chrome... Is there a third interpretatioon of chunks? – Steffen Heil Sep 19 '12 at 11:58
  • File uploading, in the way you're perceiving it, happens at the HTTP level. It doesn't happen 'in' HTML, Javascript, etc. These just offer interfaces to that functionality. All chunking/transfer can be perceived to happen at the HTTP protocol level (you can, obviously, go to a lower protocol than this but I'm guessing that would only confuse you further). – Rushyo Sep 19 '12 at 14:14

3 Answers3

1

There is a fairly young JS/HTML5 API which might cover your user case:

https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications

I can't speak to its suitability though.

Rushyo
  • 7,495
  • 4
  • 35
  • 42
  • Solved this using the File API of html5 by now. However this excludes IE up to (and even including) version 9. If there is any way to get such uploads to work in IE9, I would be glad to hear. – Steffen Heil Apr 10 '13 at 18:31
-1

If you're using IIS, the default max file upload is 4GB. You need to change this in your script or your server settings.

See: Increasing Max Upload File Size on IIS7/Win7 Pro

Community
  • 1
  • 1
Diodeus - James MacFarlane
  • 112,730
  • 33
  • 157
  • 176
-1

Normally you would break and upload such files in chunks using stream upload. If you take a limited amount data of the file, upload that part to the server, server appends data to the file. Repeat till complete file is uploaded. You can read a file in parts using FileStream (update: Adobe AIR only) or with javascript using the experimental Blob/FileReader. I guess this could be a workaround for the problem.

I think this solution could help: AS3 fileStream appears to read the file into memory

Let me know if this works out, its an interesting problem.

Community
  • 1
  • 1
Mark Knol
  • 9,663
  • 3
  • 29
  • 44