ByteBuffer's equals

Question

I have a boolean method of files comparison. It get's part of bb and check out on equal. If parts equal - get next block. If position (point) > file size and all blocks are equal - return true. Works on small files (10MB), but have troubles on big one.

private static boolean getFiles(File file1, File file2) throws IOException {
    FileChannel channel1 = new FileInputStream(file1).getChannel();
    FileChannel channel2 = new FileInputStream(file2).getChannel();
    int SIZE;
    MappedByteBuffer buffer1, buffer2;
    for (int point = 0; point < channel1.size(); point += SIZE) {
        SIZE = (int) Math.min((4096*1024), channel1.size() - point);
        buffer1 = channel1.map(FileChannel.MapMode.READ_ONLY, point, SIZE);
        buffer2 = channel2.map(FileChannel.MapMode.READ_ONLY, point, SIZE);
        if (!buffer1.equals(buffer2)) {
            return false;
        }
    }
    return true;
}

How can I modify it? Change the size of blocks?

I would try much smaller blocks probably in the range 16-128k or so... Not much more that I can think of to try :) — xpa1492, Dec 04 '14 at 09:02
see this http://stackoverflow.com/questions/964332/java-large-files-disk-io-performance — Dexter, Dec 04 '14 at 10:13
The problem is that `MappedByteBuffer` has no method for releasing the resource, instead, it relies on finalization which may happen asynchronously and deferred so when allocating buffers in a loop you might run into an `OutOfMemoryError` even when the old buffers are out of scope. I consider this a design error of the Java API, however, calling `System.gc()` occasionally might solve the problem. — Holger, Mar 18 '15 at 14:43

score 0 · Answer 1 · answered Dec 04 '14 at 09:50

0

if file2 is smaller than file1 you will get an error when trying to read data after the end of the file2, at this line:

buffer2 = channel2.map(FileChannel.MapMode.READ_ONLY, point, SIZE);

answered Dec 04 '14 at 09:50

syllabus

581
3
9

score 0 · Answer 2 · answered Dec 04 '14 at 11:06

Apart from the few corner cases that you missed, I using a Direct Allocated Byte Buffer is supposed to be faster than your method :)

public static void main (String [] args) throws IOException {

    final File file1 = new File(args[0]);
    final File file2 = new File(args[1]);

    //check if the files exist and are not blank
    if(!file1.exists() || !file2.exists() ||
        file1.length() == 0 || file2.length() == 0) {
        System.out.println("ILLEGAL FILES");
        return;
    }

    //if the length of the files is not same they are obviously not the same files
    if(file1.length() != file2.length()) {
        System.out.println("DIFFERENT SIZE");
        return;
    }

    final FileChannel channel1 = new FileInputStream(file1).getChannel();
    final FileChannel channel2 = new FileInputStream(file2).getChannel();

    //DirectByteBuffers for faster IO
    final ByteBuffer byteBuffer1 = ByteBuffer.allocateDirect(128 * 1024);
    final ByteBuffer byteBuffer2 = ByteBuffer.allocateDirect(128 * 1024);

    System.out.println("Starting Compare");

    while(true) {

        int read1, read2 =0;
        read1 = channel1.read(byteBuffer1);
        if(read1 == -1) break;

        while (read2 < read1 && read2 >= 0) {
            read2 += (channel2.read(byteBuffer2));
        }
        byteBuffer1.flip();byteBuffer2.flip();
        if(byteBuffer1.compareTo(byteBuffer2) != 0) {
            System.out.println("NOT SAME");
            return;
        }

        byteBuffer1.clear();
        byteBuffer2.clear();
    }
    System.out.println("SAME :)");
    return;
}

ByteBuffer's equals

2 Answers2