Cipher's doFinal() doesn't write bytes

Question

This is my full code:

import static java.nio.file.StandardOpenOption.READ;
import static java.nio.file.StandardOpenOption.TRUNCATE_EXISTING;
import static java.nio.file.StandardOpenOption.WRITE;

import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.MessageDigest;
import java.security.SecureRandom;
import javax.crypto.Cipher;
import javax.crypto.ShortBufferException;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public class Test {

    public static void main(String[] args) throws Exception {
        encrypt();
        decrypt();
    }

    void encrypt() throws Exception {
        Path file = Paths.get("path/to/file");
        Path backupFile = file.getParent().resolve(file.getFileName().toString() + ".bak");
        Files.deleteIfExists(backupFile);
        Files.copy(file, backupFile);

        SecureRandom secureRandom = new SecureRandom();
        byte[] initializeVector = new byte[96 / Byte.SIZE];
        secureRandom.nextBytes(initializeVector);

        Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
        GCMParameterSpec p = new GCMParameterSpec(128, initializeVector);

        try (FileChannel src = FileChannel.open(backupFile, READ);
             FileChannel dest = FileChannel.open(file, WRITE, TRUNCATE_EXISTING)) {

            SecretKeySpec secretKeySpec =
                new SecretKeySpec(MessageDigest.getInstance("MD5").digest(new byte[]{0x00}), "AES");

            cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec, p);

            ByteBuffer ivBuffer = ByteBuffer.allocate(Integer.BYTES + cipher.getIV().length);
            ivBuffer.putInt(cipher.getIV().length);
            ivBuffer.put(cipher.getIV());
            ivBuffer.flip();
            dest.write(ivBuffer);

            ByteBuffer readBuf = ByteBuffer.allocateDirect(8192);
            ByteBuffer writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(8192));
            while (src.read(readBuf) >= 0) {
                if (cipher.getOutputSize(8192) > writeBuf.capacity()) {
                    writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(8192));
                }
                readBuf.flip();

                cipher.update(readBuf, writeBuf);
                writeBuf.flip();
                dest.write(writeBuf);

                readBuf.clear();
                writeBuf.clear();
            }

            if (cipher.getOutputSize(0) > writeBuf.capacity()) {
                writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(0));
            }

            cipher.doFinal(ByteBuffer.allocate(0), writeBuf);

            writeBuf.flip();
            dest.write(writeBuf);

            Files.delete(backupFile);
        } catch (ShortBufferException e) {
            //Should not happen!
            throw new RuntimeException(e);
        }
    }

    void decrypt() throws Exception {
        Path file = Paths.get("path/to/file");
        Path backupFile = file.getParent().resolve(file.getFileName().toString() + ".bak");
        Files.deleteIfExists(backupFile);
        Files.copy(file, backupFile);

        try (FileChannel src = FileChannel.open(backupFile, READ);
             FileChannel dest = FileChannel.open(file, WRITE, TRUNCATE_EXISTING)) {

            ByteBuffer ivLengthBuffer = ByteBuffer.allocate(Integer.BYTES);
            src.read(ivLengthBuffer);
            ivLengthBuffer.flip();
            int ivLength = ivLengthBuffer.getInt();

            ByteBuffer ivBuffer = ByteBuffer.allocate(ivLength);
            src.read(ivBuffer);
            ivBuffer.flip();
            byte[] iv = new byte[ivBuffer.limit()];
            ivBuffer.get(iv);

            Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
            GCMParameterSpec p = new GCMParameterSpec(128, iv);

            SecretKeySpec secretKeySpec =
                new SecretKeySpec(MessageDigest.getInstance("MD5").digest(new byte[]{0x00}), "AES");

            cipher.init(Cipher.DECRYPT_MODE, secretKeySpec, p);

            ByteBuffer readBuf = ByteBuffer.allocateDirect(8192);
            ByteBuffer writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(8192));
            while (src.read(readBuf) >= 0) {
                if (cipher.getOutputSize(8192) > writeBuf.capacity()) {
                    writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(8192));
                }
                readBuf.flip();
                cipher.update(readBuf, writeBuf);

                writeBuf.flip();
                dest.write(writeBuf);

                readBuf.clear();
                writeBuf.clear();
            }

            if (cipher.getOutputSize(0) > writeBuf.capacity()) {
                writeBuf = ByteBuffer.allocateDirect(cipher.getOutputSize(0));
            }
            cipher.doFinal(ByteBuffer.allocate(0), writeBuf);
            writeBuf.flip();
            dest.write(writeBuf);

            Files.deleteIfExists(backupFile);
        }
    }

}

I found a strange issue: if the original file (unencrypted) is bigger than 4KB, upon decrypting, cipher.update(readBuf, writeBuf) will write nothing to the buffer, cipher.doFinal(ByteBuffer.allocate(0), writeBuf) also write nothing, and finally I get my data lost. Every calling to cipher.getOutputSize(8192), increases the result, I don't know why it happen but it may help.

Why is it happening and how can I fix it?

Could you expand your code to runnable snippet that reproduces the issue? E.g. initialize `src` and `dest` to `ByteArray(Input|Output)Stream`s with data which reproduces the problem. — Mark Peters, Oct 24 '18 at 15:22

score 0 · Accepted Answer · answered Oct 25 '18 at 06:38

.update() is easy; SunJCE implements the GCM (and CCM) requirement that authenticated decryption not release (any) plaintext if the authentication fails; see How come putting the GCM authentication tag at the end of a cipher stream require internal buffering during decryption? and https://moxie.org/blog/the-cryptographic-doom-principle/ . Because the tag is at the end of the ciphertext, this means it must buffer all the ciphertext until (one of the overloads of) doFinal() is called. (This is why for a large file your reallocation of writeBuf to cipher.getOutputSize(8192) keeps growing as you keep reading and buffering more data.)

.doFinal() is harder; it is supposed to work. However, I've narrowed down the failure: it only happens when you use ByteBuffers not raw byte[] arrays -- which is implemented in javax.crypto.CipherSpi.bufferCrypt rather than dispatching to the implementation class; and the output ByteBuffer has no backing array (i.e. was direct-allocated); and the plaintext is more than 4096 bytes. I'll try to look deeper into why this fails, but in the meantime changing either of the first two fixes it (or limiting your data to 4096 bytes, but presumably you don't want that).

Cipher's doFinal() doesn't write bytes

1 Answers1