-1

I am running a simple server app to receive uploads from a fine-uploader web client. It is based on the fine-uploader Java example and is running in Tomcat6 with Apache sitting in front of it and using ProxyPass to route the requests. I am running into an occasional problem where the upload gets to 100% but ultimately fails. In the server logs, as well as on the client, I can see that Apache is timing out on the proxy with a 502 error.

After trying and seeing this myself, I realized the problem occurs with really large files. The Java server app was taking longer than 30 seconds to reassemble the chunks into a single file and so Apache would kill the connection and stop waiting. I have increased Apache Timeout to 300 seconds which should largely correct the problem but the potential remains.

Any ideas on other ways to handle this so that the connection between Apache and Tomcat is not killed while the app is assembling the chunks on the server? I am currently using 2 MB chunks and was thinking maybe I should use a larger chunk size. Perhaps with fewer chunks to assemble the server code could do it faster. I could test that but unless the speedup is dramatic it seems like the potential for problems remain and will just be waiting for a large enough upload to come along to trigger them.

Mark Phippard
  • 10,329
  • 2
  • 32
  • 42
  • possible duplicate of [Bad Gateway 502 error with Apache mod\_proxy and Tomcat](http://stackoverflow.com/questions/169453/bad-gateway-502-error-with-apache-mod-proxy-and-tomcat) – Mark Feltner May 06 '14 at 19:11
  • I do not think so. I understand why the Timeout is happening, I am asking Fineuploader for strategy on avoiding it. Assume the server really needs 10 minutes to assemble a file from all of the chunks it was sent, short of configuring the timeout to > 10 minutes how can the client and server connection be changes? – Mark Phippard May 06 '14 at 19:29
  • Really all you can do with Fine Uploader is change the chunk size. The problem is clearly on the server. Another option not mentioned by @Ray below would be to use Fine Uploader S3. S3 is a scalable, available, and low-latency data-store and the chances of it timing out when combining the chunks of even multi-GB files is extremely low based on my experience. – Mark Feltner May 06 '14 at 22:28

1 Answers1

2

It seems like you have two options:

  1. Remove the timeout in Apache.
  2. Delegate the chunk-combination effort to a separate thread, and return a response to the request as soon as possible.

With the latter approach, you will not be able to let Fine Uploader know if the chunk combination operation failed, but perhaps you can perform a few quick sanity checks before responding, such as determining if all chunks are accessible.

There's nothing Fine Uploader can do here, the issue is server side. After Fine Uploader sends the request, its job is done until your server responds.

As you mentioned, it may be reasonable to increase the chunk size or make other changes to speed up the chunk combination operation to lessen the chance of a timeout (if #1 or #2 above are not desirable).

Ray Nicholus
  • 19,538
  • 14
  • 59
  • 82
  • I do not think you can remove the timeout, only make it impractically high. This is the answer I more or less expected was just looking for confirmation. – Mark Phippard May 07 '14 at 00:10