23

I need to somehow implement an ability to upload files through an ASP.NET application which is working within our corporate network. The problem is those files are getting increasingly big. At the moment we're using a very generic asynchronous upload but the problem is that files are getting increasingly big and the max limit of 3.9gb per file set through maxAllowedContentLength since the max value of uint won't allow anything more. Soon files which users are suppose to upload will exceed this value and might reach up to 100gb in size.

I tried looking online for some solution to this problem but in most articles by large files people mean 1gb at best.

So is there any way to upload really large files (up to 100g) through ASP.NET MVC\WebAPI application or I need to look for alternative solutions?

Cœur
  • 37,241
  • 25
  • 195
  • 267
user3223738
  • 447
  • 1
  • 4
  • 12
  • @Kamo This is actually plan B. The problem is that our users have to fill a certain form prior uploading a file plus to teach them how to use FTP would take too much time so we'd have to write a custom FTP client for that. But before doing that I'd like to explore all possibilities to modify our existing solution. – user3223738 May 08 '15 at 08:36
  • @user3223738 You could programmatically send the files over FTP? – Jamie Rees May 08 '15 at 08:41
  • @user3223738 can you ask the users to copy their files to shared folder after filling the form ? path of shared folder can be shown after the form filing. – Amitd May 08 '15 at 08:42
  • @Amitd this was also discussed and due to the fact that while we're in the same VLAN but users are located allover the country (fast food chain) there are some issues with shared folders because of that so our network admin didn't approve this idea. – user3223738 May 08 '15 at 08:46
  • 2
    I think the alternative I'd be looking at is *reducing the size of that file*. – Damien_The_Unbeliever May 08 '15 at 08:57
  • @Damien_The_Unbeliever, OP works for a fast food chain; "Super Size Me"! is their battle-cry. – DeanOC May 08 '15 at 09:07
  • Wait a second, whats wrong with the idea of splitting the file in to chunks and uploading it via ajax? – misha130 May 08 '15 at 09:26
  • Honestly never tried it(Thats why I dont have too much info) but I like this idea. http://stackoverflow.com/questions/12055386/split-file-with-javascript-or-jquery. Basically reading a file as a stream with javascript FormData library and sending every or so couple of bytes to the server and writting them to a file when the server recives it. Anyway using this you can actually make a bunch of small connections instead of one huge one. – misha130 May 08 '15 at 10:55
  • Are there any side-channels for the movement of data, out of curiosity? Do you absolutely have to go through ASP.NET? Could the user copy the file to a network file-share and provide the UNC path to the file? Or the user directly upload to cloud storage (like AWS S3 or Azure Blob storage)? – Dai Mar 25 '19 at 04:00

2 Answers2

9

Yes there is, you need to split the file in smaller parts see the example here: http://forums.asp.net/t/1742612.aspx?How+to+upload+a+big+file+in+Mvc+

Radin Gospodinov
  • 2,313
  • 13
  • 14
4

You could consider sending it in chunks. This would skip over the large file requirement (as each request would only be the size of the chunk you send), but is slightly more complicated on the client and server side.

I've done something similar for streaming uploaded files over a websocket, but this could easily be done with multiple ajax requests. In either case you'll want to use the JavaScript File API to read a segment of the file on the client's computer, encode that segment to something you can send (probably Base64), and send that particular segment to the web server. You could also send additional data such as file position to ensure the server is writing the file properly. The server can choose how to respond (can be as simple as a "true" to acknowledge receipt), after which the client javascript would read and send the next chunk of the file.

I have a demo of this using WebSockets on a github repo here (ASP.NET MVC server-side code here) but with a few tweaks you could easily make this into sequential AJAX requests.

RTigger
  • 1,360
  • 10
  • 11