Java - modify file in parts

Question

I am working on an AES 256 bit File encryption tool. The way it works now is by:

reading an entire file into a String, encrypting the string, deleting the old file, then outputting the encrypted String into the old File object.

This works for small Files, but if you were to try to encrypt a Large file, such as one larger than the Heap Space java has access too, you receive errors. Long story short, how can i read the file one MB at a time, as a String, encrypt that string, then write that MB to a temp file? This would allow for large files to be encrypted.

That is fundamentally tricky. CBC makes that impossible, and ECB is not secure. http://en.wikipedia.org/wiki/Disk_encryption_theory — SLaks, Dec 24 '12 at 14:30
You had better write the encrypted result to a _new file_, and if the new file is successfully created then delete the original file and rename the new file to the original one. This way you are guaranteed not to lose the original should writing to the new file fails. — fge, Dec 24 '12 at 14:30
thats a better idea, ill do that. but the main question is how to read the file to a string in parts — Primm, Dec 24 '12 at 14:31
Try this http://stackoverflow.com/questions/11110153/java-reading-file-chunk-by-chunk — Vishwas Shashidhar, Dec 24 '12 at 14:31
@DonutAvenger: instead of reading a file into memory, you can map it. This implies using nio, though. — fge, Dec 24 '12 at 14:32
Can you try [this](http://stackoverflow.com/a/11300687/1060037) — Jayamohan, Dec 24 '12 at 14:32
[This answer](http://stackoverflow.com/a/5623638/697630) might contain some info on NIO mapped buffers. — Edwin Dalorzo, Dec 24 '12 at 14:41
Yep, [memory mapped IO](http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/MappedByteBuffer.html) is what I'd recommend too. — JimmyB, Dec 24 '12 at 15:21

Val · Accepted Answer · 2012-12-24T15:03:38.790

3

No need for temp files. Just use File Streams (FileInputStream and FileOutputStream). Streaming I/O was exactly created for processing long files in chunks. You can use FileReader/Writer if you want to read/write strings. There is an example:

  public void decrypt(File in, File out) throws IOException, InvalidKeyException {
    aesCipher.init(Cipher.DECRYPT_MODE, aeskeySpec);
    CipherInputStream is = new CipherInputStream(new FileInputStream(in), aesCipher);
    FileOutputStream os = new FileOutputStream(out);

    int i;
    byte[] b = new byte[1024];
    while((i=is.read(b))!=-1) {
            os.write(b, 0, i);
    }

edited Dec 24 '12 at 15:03

answered Dec 24 '12 at 14:43

Val

1
8
40
64

I ended up using a FileReader to read in chars, then add them to a String. From there I encrypted that String and wrote it back. Doesnt seem very efficient though. Ill try taking a deeper look at your example. – Primm Dec 24 '12 at 15:19
1

Concatenating everything in memory defeats the very idea of streaming (in-block processing). Do you see that in.read(block) is interleaved with os.write(block)? Where do you see concatenation of the strings/blocks? – Val Dec 24 '12 at 15:26
Please make sure you use the correct encryption mode, and don't forget to include an IV. Furthermore, don't forget that `CipherInputStream` has the unfortunate side effect of ignoring padding exceptions. Some kind of cryptographic checksum (MAC or HMAC) may also be a good idea. – Maarten Bodewes Dec 24 '12 at 16:37

Java - modify file in parts

1 Answers1