Java Out of Memory Error during Encryption

Question

I am using AES to encrypt files. The problem first came when i tried to encrypt a large file. So i did some reading online and figured that i need to use a buffer and only encrypt bytes of data at a time.

I divided my plaintext into chunks of 8192 bytes of data and then applied the encryption operation on each of these chunks but I am still getting the out of memory error.

public static  File encrypt(File f, byte[] key) throws Exception
{
    System.out.println("Starting Encryption");
    byte[] plainText = fileToByte(f);
    SecretKeySpec secretKey = new SecretKeySpec(key, ALGORITHM);
    Cipher cipher = Cipher.getInstance(ALGORITHM);
    cipher.init(Cipher.ENCRYPT_MODE, secretKey);

    System.out.println(plainText.length);

    List<byte[]> bufferedFile = divideArray(plainText, 8192);


    System.out.println(bufferedFile.size());

    List<byte[]> resultByteList = new ArrayList<>();

    for(int i = 0; i < bufferedFile.size(); i++)
    {
        resultByteList.add(cipher.doFinal(bufferedFile.get(i)));
    }

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    for(byte[] b : resultByteList)
        baos.write(b);

    byte[] cipherText = baos.toByteArray();

    File temp = byteToFile(cipherText, "D:\\temp");

    return temp;

}

The fileToByte() takes a file as input and returns a byte array

the divideArray() takes a byte array as input and divides it into an arraylist consisting of smaller byte arrays.

public static List<byte[]> divideArray(byte[] source, int chunkSize) {

    List<byte[]> result = new ArrayList<byte[]>();
    int start = 0;
    while (start < source.length) {
        int end = Math.min(source.length, start + chunkSize);
        result.add(Arrays.copyOfRange(source, start, end));
        start += chunkSize;
    }

    return result;
}

Here is the error I get

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at java.io.OutputStream.write(OutputStream.java:75)
at MajorProjectTest.encrypt(MajorProjectTest.java:61)
at MajorProjectTest.main(MajorProjectTest.java:30)

I am not getting this error if I use a file of a smaller size, but then again, the sole purpose of using buffers was to eliminate the out of memory problem.

Thanks in advance. Any help is appreciated.

Answer 1

One problem is holding arrays and copies of arrays in memory.

Read and write in blocks.

Then doFinal should not be repeated. Use update instead. Many examples just use a single doFinal which is misleading.

So:

public static  File encrypt(File f, byte[] key) throws Exception
{
    System.out.println("Starting Encryption");
    SecretKeySpec secretKey = new SecretKeySpec(key, ALGORITHM);
    Cipher cipher = Cipher.getInstance(ALGORITHM);
    cipher.init(Cipher.ENCRYPT_MODE, secretKey);

    System.out.println(plainText.length);

    Path outPath = Paths.get("D:/Temp");
    byte[] plainBuf = new byte[8192];
    try (InputStream in = Files.newInputStream(f.toPath());
            OutputStream out = Files.newOutputStream(outPath)) {
        int nread;
        while ((nread = in.read(plainBuf)) > 0) {
            byte[] enc = cipher.update(plainBuf, 0, nread);
            out.write(enc);
        }       
        byte[] enc = cipher.doFinal();
        out.write(enc);
    }
    return outPath.toFile();
}

---

Explanation

Encryption of some byte blocks goes as:

• Cipher.init
• Cipher.update block[0]
• Cipher.update block[1]
• Cipher.update block[2]
• ...
• Cipher.doFinal(block[n-1])

Or instead of the last doFinal:

• Cipher.update(block[n-1])
• Cipher.doFinal()

Every update or doFinal yielding a portion of the encrypted data.

doFinal also "flushes" final encryption data.

If one has only a single block of bytes, it suffices to call

byte[] encryptedBlock = cipher.doFinal(plainBlock);

Then no calls to cipher.update are needed.

For the rest I used the try-with-resources syntax which automatically closes input and output streams, even should a return happen, or an exception have been thrown.

Instead of File the newer Path is a bit more versatile, and in combination with Paths.get("...") and the very nice utility class Files can provide powerful code: like Files.readAllBytes(path) and much more.

Answer 2

Look at these four variables: plainText, bufferedFile, resultByteList, cipherText. All of them contain your entire file in a slightly different format, which means that each of them is 1.2GB big. Two of them are Lists which means they are likely to be even bigger, because you didn't set the initial size of ArrayLists and they resize automatically when needed. So we are talking about over 5GB of memory needed.

Actually, you add chunks into ByteArrayOutputStream baos, which means that it must store it internally, before you call toByteArray() on it. So it's 5 copies of your data, meaning 6GB+. The ByteArrayOutputStream is internally using an array so it grows similarly to ArrayLists so it will use more memory than needed (see the stacktrace - it tried to resize).

All these variables are in the same scope, never are assigned null which means that they cannot be garbage collected.

---

You can increase the maximum heap limit (see Increase heap size in Java), but this will be a serious limitation on your program.

Your program throws out of memory error when writing to ByteArrayOutputStream. This is the 4th time you copy all your data, which means that 3.6GB is already allocated. From this I deduce that your heap is set to 4GB (which is a maximum you can set on 32 bit operating system).

---

What you should do is have a loop, read part of the file, encrypt it and write to another file. This will avoid loading entire file into memory. Lines like List<byte[]> bufferedFile = divideArray(plainText, 8192); or resultByteList.add(...) is something that you shouldn't have in your code - you end up storing entire file in memory. The only thing that you need to keep track of is a cursor (i.e. position which says what bytes you already processed), which is O(1) memory complexity. Then you only need as much memory as the chunk your are encoding - which is far smaller than entire file.

Answer 3

As you're iterating over the file, keep a counter to track the number of bytes:

int encryptedBytesSize = 0;
for(int i = 0; i < bufferedFile.size(); i++) {
    resultByteList.add(cipher.doFinal(bufferedFile.get(i)));
    encryptedBytesSize += resultByteList.get(resultByteList.size() - 1).length;
}

Then use the constructor which takes a size parameter to create the output buffer:

ByteArrayOutputStream baos = new ByteArrayOutputStream(encryptedBytesSize);

This will avoid the internal buffer from having to grow. Growth could be non-linear so as more bytes are added each iteration even more space is allocated the next time it grows.

But this still might not work, depending on the file size. Another approach would be to:

1. Read a little chunk of the unencrypted file
2. Encrypt the chunk
3. Write to the encrypte file

This avoids having all of the regular and encrypted files in memory at the same time.