How to chain-encrypt using AES CBC mode without writing to file

Question

I need to encrypt a large contiguous amount of bytes, not fitting into single byte array.

There are already a few similar questions, but none of the answers works for me:

stackoverflow.com/questions/5090233/how-to-encrpyt-decrypt-data-in-chunks stackoverflow.com/questions/45735983/encrypting-files-with-aes-c-sharp-in-chunks stackoverflow.com/questions/27645527/aes-encryption-on-large-files

Following code works if the output length is under limit (Key and IV are hardcoded for testing):

byte[] encrypt(byte[] input, byte[] iv = null)
{
    var aes = new AesManaged();
    aes.KeySize = 128;
    aes.Key = new byte[16] { 0x0F, 0xD4, 0x33, 0x82, 0xF4, 0xDF,
        0x62, 0xA5, 0x55, 0x7C, 0x6E, 0x92, 0xC5, 0x64, 0x67, 0xA9 };
    aes.IV = (iv != null) ? iv :
        new byte[16] { 0xB3, 0x87, 0xDA, 0xA0, 0x47, 0x7C,
        0x52, 0x76, 0xCB, 0x3A, 0x69, 0x9B, 0x0F, 0x82, 0xAF, 0xA7 };

    using (var stream = new MemoryStream())   // size limit is 2 GB
    using (var cryptoStream = new CryptoStream(stream,
            aes.CreateEncryptor(), CryptoStreamMode.Write))
    {
        cryptoStream.Write(input, 0, input.Length);
        cryptoStream.FlushFinalBlock();
        return stream.ToArray();    // max number of bytes = int.MaxValue
    }
}

One alternative is to use FileStream instead of MemoryStream, but I prefer to get the results in memory.

I am trying to implement the chain operation, dividing the input on chunks of [multiples of AES BlockSize] and capture the encrypted output after feeding each chunk.

According to what I read, the last bytes of each result are supposed to be used as an IV for the next encryption, but that does not work. For simplicity, chunk size is equal to BlockSize:

byte[] subArray(byte[] source, int start, int length = -1)
{
    if (length == -1) length = source.Length - start;
    var target = new byte[length];
    Buffer.BlockCopy(source, start, target, 0, length);
    return target;
}
var encr0 = encrypt(subArray(input, 0,         blockSize * 2));
var encr1 = encrypt(subArray(input, 0,         blockSize));
var encr2 = encrypt(subArray(input, blockSize, blockSize),
                    subArray(encr1, blockSize, 16));
Assert.IsTrue(encr2.SequenceEqual(subArray(encr0, blockSize)));  // FAILS

What I am doing wrong?

You could also just create your own version of MemoryStream that uses chunks instead of a single array... — bartonjs, Jan 23 '23 at 19:30
@bartonjs, the question was about discrepancy of every block after the first with expected result. I made two mistakes: assumed BlockSize = 128 bytes, when it was in bits and must be 16 bytes, and messed up with padding. Answer below is accurate. — Andrei Kalantarian, Jan 24 '23 at 20:14

Topaco · Accepted Answer · 2023-01-22T22:11:53.963

2

The encrypt() method uses the CBC mode by default. In this mode, the ciphertext of a block serves as IV of the following block (a random IV is applied for the first block).
In the posted code, decryption fails because an incorrect IV is used for encr2. The correct IV is subArray(encr1, 0, blockSize) (namely the ciphertext of the first block) and not subArray(encr1, blockSize, 16).
When this is fixed, the exception no longer occurs.

Another issue is that encrypt() implicitly performs PKCS#7 padding (the default). PKCS#7 padding pads even if the last plaintext block is completely filled. In this case, a full padding block (consisting of 16 0x10 values for AES) is appended.
The padding in encrypt() causes plaintext blocks that are not at the end to be incorrectly padded with a full padding block as well. This results in a wrong ciphertext block, as can be easily checked when encr0, encr1 and encr2 (e.g. hex encoded) are output.

To avoid these wrong padding blocks, the padding in encrypt() must be disabled for all plaintext blocks except the last one which is regulary padded (if it is incomplete it is filled, if it is complete, a full padding block is appended).

Edit: For arbitrary chunksizes (of the size of a multiple of the blocksize) subArray(encr1, chunkSize - blockSize, blockSize) must be applied as IV for encr1. If the chunksize is equal to the blocksize (as assumed here) this simplifies to subArray(encr1, 0, blockSize).

edited Jan 22 '23 at 22:11

answered Jan 22 '23 at 12:33

Topaco

40,594
4
35
62

Code with fixed IV but with padding issue: https://dotnetfiddle.net/QlhSob – Topaco Jan 22 '23 at 12:55
1

Code with fixed IV and fixed padding issue: https://dotnetfiddle.net/GWaJ0B – Topaco Jan 22 '23 at 12:55
thanks, I got it working with any number of chunks and any size of a chunk (multiples of BlockSize). Only correction: IV size is fixed and has to be 1/8 of BlockSize, or 16: subArray(encr1, 0, 16). Lat 16 bytes were generated because of padding mode, and I was erroneously thinking, that it is IV for the next chunk. Thanks again! – Andrei Kalantarian Jan 22 '23 at 14:29
@Topako, I was taking BlockSize from AesManaged, and it was 128, not 16. Sorry for the confusion, the encryption terminology is new to me. Size of the whole chunk could be multiple of 16 (not necessarily 128). IV for the next chunk is the last 16 bytes of previous chunk. Your answer and the code samples are greatly appreciated! – Andrei Kalantarian Jan 22 '23 at 16:10
1

@AndreiKalantarian - I assumed a chunksize of 16 bytes (= blocksize), because the question stated *chunk size is equal to BlockSize*. But in the end the statements made are independent of the value of the chunksize (assuming a multiple of the blocksize). Only the IV for `encr1` has to be adapted, of course, see the edit section and this code for a chunksize of 128 bytes: https://dotnetfiddle.net/hpFHLj. – Topaco Jan 22 '23 at 18:18
@Topako, I simply forgot to include BlockSize in the question. After posting, considered to add it, but thought that it has to be one of the "universal constants" (aes.BlockSize returns 128), and didn't realize it is in bits. Sorry about that! – Andrei Kalantarian Jan 22 '23 at 23:32

How to chain-encrypt using AES CBC mode without writing to file

1 Answers1