22

I have a requirement of uploading a large file over HTTP to a remote server. I am researching on how to send the data using multipart/form-data.

I have gone through How does HTTP file upload work? and understood how it separates the file data using boundaries.

I wanted to know whether all the file data is sent at one go or is streamed with several requests to the remote server.

Because if it is sent at one go, it is not possible to read the whole data at the remote server and write it to a file.

But if it streamed, how does the remote server parses the streamed data, write this streamed data to a file and redo the same thing till all the data is streamed.

Sorry if it a noob question, I am researching about it as well.

Maybe it is outside the scope of multipart/form-data and HTTP is itself taking care of.

Any help is appreciated.

dreftymac
  • 31,404
  • 26
  • 119
  • 182
Rohit Ranjan Verma
  • 221
  • 2
  • 7
  • What do you mean by "at one go"? One file definitely isn't sent through multiple requests. What do you mean by _"if it is sent at one go, it is not possible to read the whole data at the remote server"_? It all depends on the web server and the application running behind it. – CodeCaster Dec 11 '18 at 14:13
  • What I mean by 'at one go' is that the whole file data is sent in the request body or the file data is divided into several chunks and then each chunk is streamed to the remote server. Sorry for the confusion, in my case, the server does not have the capability to read the whole data at one go due to memory limitations. What I am also confused about even if it is streamed in chunks(idk), the remote server will have to combine these chunks and then write the whole data to a file leading to keeping it in memory and therefore not suitable in this use case. Sorry if it is a noob question. – Rohit Ranjan Verma Dec 11 '18 at 14:32
  • What kind of server application are you using? Files are not send in multiple requests. Files are broken down to chunks in one request stream. Depending on the size of the file and server acceptable criteria on max size, it can be a small number chunks or thousands of chunks sent through a stream in a single request. – Luis Estevez Dec 11 '18 at 17:16
  • The remote server can use nginx + php-fpm or apache. What I wanted to know was whether multipart/form-data automatically divides the file data into chunks and sends it one request stream or it has to done manually, or there is some standard way to do that. – Rohit Ranjan Verma Dec 12 '18 at 05:16
  • https://stackoverflow.com/questions/12871362/streaming-data-of-unknown-size-from-client-to-server-over-http-in-python The above link provides a way to send the file data in chunks over a single HTTP connection. Does multipart/form-data does it the same way? Also, if I want to write the data at the server, is it possible to write the first chunk data to a file and then append the later chunks to the same file using multipart/form-data? Hope this is not confusing. – Rohit Ranjan Verma Dec 12 '18 at 07:24
  • see if this helps: https://gist.github.com/CMCDragonkai/6bfade6431e9ffb7fe88 – h.i Jul 03 '19 at 14:36

1 Answers1

-1

The logistics of the sending is not relevant. What matters is the maximum request size that is set on the server side. How it is set depends on the technology used there: IIS, Apache, nginx? If the post request of the browser exceeds that size (because of a too large file), errors will happen. There is nothing on the browser side u can tweak or change to fix breaking uploads. Unless you are building your own browser:-)

dreftymac
  • 31,404
  • 26
  • 119
  • 182
webjazznl
  • 1
  • 2