Fastest way to copy files in Java

Question

What ist the fastest way to copy a big number of files in Java. So far I have used file streams and nio. Overall streams seem to be faster than nio. What experiences did you make so far?

Answer 1

http://www.baptiste-wicht.com/2010/08/file-copy-in-java-benchmark/ might get you your answer.

For the benchmark, I made the tests using different files.

1. Little file (5 KB)
2. Medium file (50 KB)
3. Big file (5 MB)
4. Fat file (50 MB)
5. And an enormous file (1.3 GB) only binary

And I made the tests first using text files and then using binary files. I made the tests using in three modes :

1. On the same hard disk. It's an IDE Hard Disk of 250 GB with 8 MB of cache. It's formatted in Ext4.
2. Between two disk. I used the first disk and an other SATA Hard Disk of 250 GB with 16 MB of cache. It's formatted in Ext4.
3. Between two disk. I used the first disk and an other SATA Hard Disk of 1 TB with 32 MB of cache. It's formatted using NTFS.

I used a benchmark framework, described here, to make the tests of all the methods. The tests have been made on my personal computer (Ubuntu 10.04 64 bits, Intel Core 2 Duo 3.16 GHz, 6 Go DDR2, SATA Hard Disks). The Java version used is a Java 7 64 bits Virtual Machine...

Answer 2

I would use:

import java.io.*;
import java.nio.channels.*;

public class FileUtils{
    public static void copyFile(File in, File out) 
        throws IOException 
    {
        FileChannel inChannel = new
            FileInputStream(in).getChannel();
        FileChannel outChannel = new
            FileOutputStream(out).getChannel();
        try {
            inChannel.transferTo(0, inChannel.size(),
                    outChannel);
        } 
        catch (IOException e) {
            throw e;
        }
        finally {
            if (inChannel != null) inChannel.close();
            if (outChannel != null) outChannel.close();
        }
    }

    public static void main(String args[]) throws IOException{
        FileUtils.copyFile(new File(args[0]),new File(args[1]));
  }
}

If any of your files are bigger than 64M in Windows you might need to look at this: http://forums.sun.com/thread.jspa?threadID=439695&messageID=2917510

Answer 3

Using Stream

private static void copyFileUsingStream(File source, File dest) throws IOException {
    InputStream is = null;
    OutputStream os = null;
    try {
        is = new FileInputStream(source);
        os = new FileOutputStream(dest);
        byte[] buffer = new byte[1024];
        int length;
        while ((length = is.read(buffer)) > 0) {
            os.write(buffer, 0, length);
        }
    } finally {
        is.close();
        os.close();
    }
}

Using Channel

private static void copyFileUsingChannel(File source, File dest) throws IOException {
    FileChannel sourceChannel = null;
    FileChannel destChannel = null;
    try {
        sourceChannel = new FileInputStream(source).getChannel();
        destChannel = new FileOutputStream(dest).getChannel();
        destChannel.transferFrom(sourceChannel, 0, sourceChannel.size());
       }finally{
           sourceChannel.close();
           destChannel.close();
       }
}

Using Apache Commons IO

private static void copyFileUsingApacheCommonsIO(File source, File dest) throws IOException {
    FileUtils.copyFile(source, dest);
}

Using Java SE 7 Files

private static void copyFileUsingJava7Files(File source, File dest) throws IOException {
    Files.copy(source.toPath(), dest.toPath());
}

Performance test

File source = new File("/Users/tmp/source.avi");
File dest = new File("/Users/tmp/dest.avi");

//copy file conventional way using Stream
long start = System.nanoTime();
copyFileUsingStream(source, dest);
System.out.println("Time taken by Stream Copy = "+(System.nanoTime()-start));

//copy files using java.nio FileChannel
source = new File("/Users/tmp/sourceChannel.avi");
dest = new File("/Users/tmp/destChannel.avi");
start = System.nanoTime();
copyFileUsingChannel(source, dest);
System.out.println("Time taken by Channel Copy = "+(System.nanoTime()-start));

//copy files using apache commons io
source = new File("/Users/tmp/sourceApache.avi");
dest = new File("/Users/tmp/destApache.avi");
start = System.nanoTime();
copyFileUsingApacheCommonsIO(source, dest);
System.out.println("Time taken by Apache Commons IO Copy = "+(System.nanoTime()-start));

//using Java 7 Files class
source = new File("/Users/tmp/sourceJava7.avi");
dest = new File("/Users/tmp/destJava7.avi");
start = System.nanoTime();
copyFileUsingJava7Files(source, dest);
System.out.println("Time taken by Java7 Files Copy = "+(System.nanoTime()-start));

RESULTS

Time taken by Stream Copy            =  44,582,575,000
Time taken by Java7 Files Copy       =  89,061,578,000
Time taken by Channel Copy           = 104,138,195,000
Time taken by Apache Commons IO Copy = 108,396,714,000

Answer 4

Have java fork off an OS batch script that copies the files. Your code might have to write the batch script.

Answer 5

You can use either FileUtils implementation of apache commons-io library to copy file

FileUtils.copyFile(new File(sourcePath), new File(destPath));

Which uses FileChannel for IO operation.

Or use java.nio.file.Files's copy() method.

Answer 6

Its depend of files(bigger files), for me is fastest way this one with buffered stream

 public void copyFile(File inFileStr, File outFileStr) throws IOException {

    try (BufferedInputStream bis = new BufferedInputStream(new FileInputStream(inFileStr)); 
        BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(outFileStr))) {

        byte[] buffer = new byte[1024 * 1024];
        int read = 0;
        while ((read = bis.read(buffer)) != -1) {
            bos.write(buffer, 0, read);
        }

        bis.close();
        bos.close();
    } catch (IOException ex) {
        ex.printStackTrace();
    }

}