After decrypting EVP cipher with OpenSSL, 8 bytes of plaintext are always incorrect

Question

I'm having trouble decrypting some data with the OpenSSL library, using the EVP functions and a symmetric key. I encrypt the data on the command like using openssl enc, then decrypt with come C++ code. This works ... mostly.

No matter what data I use, after I've performed the decryption, the second block of 8 bytes in the plaintext isn't correct (bytes 8 through 15) - but the rest of the file is. I've even done this with a 130 meg file - and all 130 megs are perfectly correct and in the correct position in the file, except for those bytes.

This happens on both out ARM target, and when built on Ubuntu 12.04 (different libraries, different toolchains).

Here is a short, complete program that has the problem. Below that is some terminal output that demonstrates it.

#include <string>
#include <fstream>
#include <stdexcept>
#include <openssl/evp.h>

void decrypt_and_untar(const std::string& infile, const std::string& outfile)
{
    unsigned char key[] = {0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff,
                           0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xaa, 0xbb, 0xcc, 0xdd, 0xee, 0xff};
    unsigned char iv[] = {0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77};

    std::ifstream is(infile, std::ios_base::in | std::ios_base::binary);
    std::ofstream os(outfile, std::ios_base::out | std::ios_base::binary);

    auto ctx = EVP_CIPHER_CTX_new();

    if (EVP_DecryptInit_ex(ctx, EVP_aes_256_cbc(), nullptr, key, iv))
    {
        const int BufferSize = 10240;
        unsigned char enc_buff[BufferSize];
        int bytes_in_enc_buff = 0;
        unsigned char dec_buff[BufferSize + EVP_CIPHER_CTX_block_size(ctx)];
        int bytes_in_dec_buff = 0;

        while (!is.eof())
        {
            is.read(reinterpret_cast<char*>(enc_buff), BufferSize);
            bytes_in_enc_buff = is.gcount();
            if (bytes_in_enc_buff > 0)
            {
                if (EVP_DecryptUpdate(ctx, dec_buff, &bytes_in_dec_buff, enc_buff, bytes_in_enc_buff))
                {
                    os.write(reinterpret_cast<char*>(dec_buff), bytes_in_dec_buff);
                    bytes_in_enc_buff = 0;
                    bytes_in_dec_buff = 0;
                }
                else
                    throw std::runtime_error("Failed DecryptUpdate.");
            }
        }

        if (EVP_DecryptFinal_ex(ctx, dec_buff, &bytes_in_dec_buff))
            os.write(reinterpret_cast<char*>(dec_buff), bytes_in_dec_buff);
        else
            throw std::runtime_error("Failed DecryptFinal.");
    }
    else
    {
        throw std::runtime_error("Failed DecryptInit.");
    }


    EVP_CIPHER_CTX_free(ctx);
}

int main(int argc, char* argv[])
{
    if (argc == 3)
        decrypt_and_untar(argv[1], argv[2]);

    return 0;
}

Here is a demo of the problem in action. I create a 1 meg file of all zeroes, encrypt it, decrypt it, and then look at it with od. If I perform this multiple times, the values for those 8 wrong bytes change from run to run...

~/workspace/test_crypto/Debug$ dd if=/dev/zero of=plain.original bs=1024 count=1024
1024+0 records in
1024+0 records out
1048576 bytes (1.0 MB) copied, 0.00154437 s, 679 MB/s
~/workspace/test_crypto/Debug$ od -t x1 plain.original 
0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
4000000
~/workspace/test_crypto/Debug$ /usr/bin/openssl enc -aes-256-cbc -salt -in plain.original -out encrypted -K 00112233445566778899AABBCCDDEEFF00112233445566778899AABBCCDDEEFF -iv 0011223344556677
~/workspace/test_crypto/Debug$ ./test_crypto encrypted plain
~/workspace/test_crypto/Debug$ od -t x1 plain
0000000 00 00 00 00 00 00 00 00 00 40 02 0d 18 93 b8 bf
0000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
4000000
~/workspace/test_crypto/Debug$

Ouch! You even knew to call `EVP_CIPHER_CTX_block_size(ctx)` to get the block size :) Next time, let the code debug itself: `ASSERT(EVP_CIPHER_CTX_block_size(ctx) == COUNTOF(iv));`. You could even do similar for the key. — jww, Jul 01 '16 at 14:08
@jww Well, the API documentation was pretty clear on that. I wish it were clear on other things as well :) Thanks. — Steve, Jul 01 '16 at 16:40

zaph · Accepted Answer · 2016-07-01T13:51:39.057

4

The iv is to short, the iv needs to be a full block length, 16-bytes for AES. That is why bytes 8-15 are incorrect, they correspond to the missing iv bytes.

In CBC mode on encryption the iv is xor'ed with the first block of plain text and on decryption xor'ed with the decrypted output. Best guess is that the encryption implementation is picking up 8-bytes of junk just past the (to short) iv so it is different on each run, both for encryption and decryption.

edited Jul 01 '16 at 13:51

answered Jul 01 '16 at 13:45

zaph

111,848
21
189
228

This was it! Thanks! – Steve Jul 01 '16 at 16:41
By the way - how would this only effect those 8 bytes? I would expect the algorithm to be "off" from that point forward, or perhaps every second 8-byte block wrong, etc... – Steve Jul 01 '16 at 18:00
1

During encryption the first block is xor'ed with the iv, the next block is xor'ed with the 1st block's encrypted data, the iv is only used for the first block. During decryption the first block is decrypted and xor'd with the iv, the first 8-bytes are correct, the 2nd are garbage so the output 2nd 8-bytes are garbage. The 2nd block is decrypted and xor'ed with the previous blocks encrypted data so it is correct. In general CBC mode will recover from a corrupt block. See the [CBC mode link](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Block_Chaining_.28CBC.29). – zaph Jul 01 '16 at 18:26

After decrypting EVP cipher with OpenSSL, 8 bytes of plaintext are always incorrect

1 Answers1