1

i would like to know which is the best way to copy large number of files.

Jonik
  • 80,077
  • 70
  • 264
  • 372
michal
  • 101
  • 3
  • 13
  • copy from where to where ? local ? over network ? – Andrew Keith Nov 10 '09 at 08:57
  • 1
    Yes, you need to elaborate. What are your goals? – Prof. Falken Nov 10 '09 at 09:12
  • i need to find a really fast method for copying lots of files from one place to another – michal Nov 10 '09 at 09:19
  • 1
    To simplify your code you could use e.g. Commons IO (see Guillaume's answer), but whichever library you choose won't change the fact that copying (as opposed to moving) a large number of files will be *slow* due to disk IO limitations. – Jonik Nov 10 '09 at 09:59
  • 1
    Oh, the NIO related answers here might be of interest to you: http://stackoverflow.com/questions/106770/standard-concise-way-to-copy-a-file-in-java – Jonik Nov 10 '09 at 11:31

4 Answers4

4

If you're copying files locally, you should use FileChannel.transferFrom() or FileChannel.transferTo(). For example:

FileInputStream is = new FileInputStream(file);
FileOutputStream fos = new FileOutputStream(oFile);
FileChannel f = is.getChannel();
FileChannel f2 = fos.getChannel();

f.transferTo(0, f.size(), f2);

f2.close();
f.close();

On many platforms, the copying will happen via DMA and be about as fast as possible.

If you're copying files over a network, then you're in a different situation. In the case that some of the files might already exist, then you should look into rsync, as it can avoid transferring parts of files which are the same on both sides. If the most common case is that the files won't already exist on the destination machine, then you'll want to use something like scp. For that, see this question.

Community
  • 1
  • 1
uckelman
  • 25,298
  • 8
  • 64
  • 82
  • +1: Gotta love that DMA. But I'm afraid most kids don't know much about it. http://en.wikipedia.org/wiki/Direct_memory_access – Stu Thompson Dec 28 '09 at 18:38
2

"Best" needs clarification.

I would delegate to rsync which is very good at copying a large number of files with a lot of options. I am not aware of a good Java implementation of rsync.

Thorbjørn Ravn Andersen
  • 73,784
  • 33
  • 194
  • 347
2

Check out the Files class from Google's guava library. It contains some utility methods for copying whole files.

finnw
  • 47,861
  • 24
  • 143
  • 221
  • Only problem is that no released version of Guava exists yet - only option is to check out trunk sources from their svn. (Correct me if I'm wrong.) – Jonik Nov 10 '09 at 09:35
  • Guava has been release for a long while now – David Jan 20 '12 at 00:25
2

As always, there is the Jakarta Commons : http://commons.apache.org/io/api-release/org/apache/commons/io/FileUtils.html

Guillaume
  • 18,494
  • 8
  • 53
  • 74
  • At the time of writing this, FileUtils.copyFile() uses IOUtils.copy(), which just reads from an InputStream and writes to an OutputStream. It doesn't do anything to detect the special case where you have a FileInputStream and a FileOutputStream and so could do a faster copy via channels. So, if the goal is to do the copy as fast as possible *and* the copy is local, then this won't be the fastest way. won't be the fastest way. – uckelman Nov 10 '09 at 11:05