6

I've implemented a REST service for downloading a moderately large (250 Mb) zip file using IOUtils.copy(), similar to Most effective way to write file to servletoutputstream. The REST service is called from another app using HttpURLConnection. I use IOUtils.copy() on the receiving side to save the file to disk.

It takes over 30 minutes to download a file. This is way too slow.

Ideas? Better implementations?

Community
  • 1
  • 1
Brett Slocum
  • 390
  • 1
  • 5
  • 21
  • 1
    Can you post the relevant code for us to inspect? – Sujay Jul 13 '12 at 20:33
  • Have you tried what was suggested by `BalusC` on the same page: http://stackoverflow.com/a/4647337/20634 ? If yes, what are the results ? – Alex Jul 13 '12 at 20:37
  • 2
    Its highly unlikely to be due to software. Its far more likely to be due to bandwidth or hardware issues. The transfer rate is 1.1 Mb/s which is a common limit for broadband upload speeds. – Peter Lawrey Jul 13 '12 at 20:38
  • I haven't implemented FileChannels from BalusC, since I've seen no indication that they go faster. – Brett Slocum Jul 16 '12 at 18:11

2 Answers2

5

I found the issue.

I was running both the client and the server on the same machine for testing. As soon as I moved one to a different machine, the transfer took a little over a minute.

Brett Slocum
  • 390
  • 1
  • 5
  • 21
0

You can read the source code and see for yourself that the default buffer size is 4096.
I personally used 8192 as buffer size in a specific case.
Consider playing with the buffer size.
You can implement this copy quite easily on your own, or see the source file here, as a reference.

In addition, consider performing multi-threaded download, if possible as your server supports multiple connections (I assume).

Francis Upton IV
  • 19,322
  • 3
  • 53
  • 57
Yair Zaslavsky
  • 4,091
  • 4
  • 20
  • 27