Split large files using Java

Question

How can I split a file into parts larger than 2GB? An array of bytes accepts an int instead of a long as the size. any solution?

public void splitFile(SplitFile file) throws IOException {
    int partCounter = 1;
    int sizeOfFiles = (int)value;
    byte[] buffer = new byte[sizeOfFiles];
    File f = file.getFile();
    String fileName = f.getName();

    try (FileInputStream fis = new FileInputStream(f);
         BufferedInputStream bis = new BufferedInputStream(fis)) {
            int bytesAmount = 0;
            while ((bytesAmount = bis.read(buffer)) > 0) {
                String filePartName = fileName + partCounter + file.config.options.getExtension();
                partCounter++;
                File newFile = new File(f.getParent(), filePartName);
                try (FileOutputStream out = new FileOutputStream(newFile)) {
                    out.write(buffer, 0, bytesAmount);

                } 
            }
        }
}

Please checkout https://stackoverflow.com/questions/19177994/java-read-file-and-split-into-multiple-files — user4617883, Dec 28 '19 at 17:22
Do not try to read the *entire file* into a single byte array. You will devastate your program’s performance. The whole point of InputStreams and OutputStreams is that you can read and write data a little at a time. — VGR, Dec 28 '19 at 17:59
Does this answer your question? [Java - Read file and split into multiple files](https://stackoverflow.com/questions/19177994/java-read-file-and-split-into-multiple-files) — Kaan, Dec 28 '19 at 19:37

score 0 · Answer 1 · answered Dec 28 '19 at 18:06

Don't read the entire file into memory, obviously, or even an entire 'part file'.

Your code as pasted will split the file into as many parts as the read method partitions; this seems very silly; after all, the read() method is specced to allow it to partition into single byte increments.

Don't make a new part-file for every call to read. Instead, separate this out: Your read call gets anywhere from 1 to <BUFFER_SIZE> bytes, and your part's size is <PART_SIZE> large; these two things do not have to be the same and you shouldn't write the code that way.

Once you have an open FileOutputStream you can call .write(buffer, 0, bytesAmount) on it any number of times; you can even call .write(buffer, 0, theSmallerOfBytesLeftToWriteInThisPartAndBytesAmount) followed by opening up the next part file FileOutputStream and calling .write(buffer, whereWeLeftOff, remainder) on that one.

Can you show me an example? – Mr Robot Dec 29 '19 at 11:41 — Mr Robot, Dec 29 '19 at 11:41

Split large files using Java

1 Answers1