1

I have the following code (modified ik and iv, of course):

#include "openssl/evp.h"

#include <iostream>
#include <sstream>
#include <string>

using std::cout;
using std::endl;
using std::string;
using std::ostringstream;

void Encryption(bool encrypt, const string &in, string &out)
{
    int ik[17] = {  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16 };
    int iv[17] = { 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32 };
    ik[16] = iv[16] = '\0';
    unsigned char ckey[17], ivec[17];
    for (int i = 0; i < 16; ++i)
    {
        ckey[i] = ik[i];
        ivec[i] = iv[i];
    }
    EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
    EVP_CipherInit(ctx, EVP_aes_256_cbc(), ckey, ivec, encrypt);
    unsigned blocksize = EVP_CIPHER_CTX_block_size(ctx);

    int out_len;
    int n = in.length() + blocksize;
    unsigned char *cipher_buf = new unsigned char[n];
    for (int i = 0; i < n; ++i)
        cipher_buf[i] = ' ';
    EVP_CipherUpdate(ctx, cipher_buf, &out_len,
                     (const unsigned char*)in.c_str(), in.length());
    std::ostringstream ss;
    ss.write((char*)cipher_buf, out_len);
    EVP_CipherFinal(ctx, cipher_buf, &out_len);
    ss.write((char*)cipher_buf, out_len);
    delete cipher_buf;
    out = ss.str();
    EVP_CIPHER_CTX_free(ctx);
}    

int main()
{
    string out;
    string in = "TEST";
    string back;
    Encryption(true, in, out);
    cout << "---------------" << endl;
    Encryption(false, out, back);
    cout << "TEST > " << out << " > " << back << endl;
    return 0;
}

Basically, this code encrypts a string, decrypts it and then prints out the result from both steps.

If I run this code in Debug, it works just fine, outputting TEST > *gibberish A* > TEST. If I run it on Release, however, it fails, printing only TEST > *gibberish B (not A)* >, because back remains empty. Seeing this question and another one I can't find now, I made a copy of my Release configuration and went changing one parameter at a time to the equivalent value in the Debug configuration. When I changed Configuration Properties > C/C++ > Code Generation > Basic Runtime Checks from Default to Both (/RTC1, equiv. to /RTCsu) (/RTC1), this new Release configuration worked, printing exactly the same values as in the Debug configuration. Obviously, by then I'd already changed many other parameters, but it was only when I changed the value of /RTC that any change was clear in the execution.

I also noticed something else. If I removed the cout << "---------------" << endl; line in main(), the default Release configuration also works, printing TEST > *gibberish C* > TEST. The encryption gibberish is different to both the Debug result and the Release with the cout line. Also, if I place that cout call anywhere other than between my Encryption calls, it works.

So I have two questions I believe may be interrelated:

  • Why does the value of /RTC affect the encryption result? Given that /RTC should be disabled in Release builds, is it possible to generate equivalent encrypted data (aka gibberish) in both Debug and Release, so as to make them interchangeable, without losing /RTC in Debug?
    • Why does the use of cout in between my Encryption() calls modify the result in Release mode and only in Release mode?

And, should it be worth mentioning, everything is built with the static runtime library (/MT[d]).

Community
  • 1
  • 1
Wasabi
  • 2,879
  • 3
  • 26
  • 48
  • Not the source of your problem, but; your code could benefit a great deal from using modern C++ - things like `auto`, smart pointers (`std::unique_ptr` & `std::shared_ptr`) and range-for. Instead of manually typing types all over the place and managing memory by hand, etc. Just a tip :) – Jesper Juhl Nov 16 '16 at 19:58
  • I would start with a Know Answer Test (KAT), likes [this one](http://csrc.nist.gov/groups/STM/cavp/documents/aes/AESAVS.pdf) from NIST. See section 6.2. That will allow you to tell which answer is correct. I *suspect* you have memory overwriting issues, and the checks are causing different bits of memory to get clobbered. Step through the broken version (probably Release), and work out where the problem lies. (You do know you can run Release code under the debugger?) – Martin Bonner supports Monica Nov 16 '16 at 19:58
  • @JesperJuhl Thanks for the tips. I actually do use such techniques (all hail range-for) in my main code. In regards to `auto` I actually changed my code to post it here to make it more explicit. And yeah, I could have used smart pointers for cipher_buf, I suppose. But how could I have used a range-for to populate cipher_buf, without an iterator? That's one trick I'd love to learn. – Wasabi Nov 16 '16 at 20:09
  • You program has problems beyond debug and release. AES has a forward and reverse direction. You should be using `EVP_EncryptInit`, `EVP_EncryptUpdate` and `EVP_EncryptFinal` for encryption; and `EVP_DecryptInit`, `EVP_DecryptUpdate` and `EVP_DecryptFinal` for decryption. Also see [EVP Symmetric Encryption and Decryption](https://wiki.openssl.org/index.php/EVP_Symmetric_Encryption_and_Decryption) on the OpenSSL wiki. – jww Nov 16 '16 at 20:30
  • @jww I'll take a look at those. I based my code on what I found on [this answer](http://stackoverflow.com/q/24856303/2175231), which only uses Encrypt. – Wasabi Nov 16 '16 at 22:24
  • @Wasabi - The question you cited uses `EVP_EncryptInit`, `EVP_EncryptUpdate` and `EVP_EncryptFinal` for encryption; and `EVP_DecryptInit`, `EVP_DecryptUpdate` and `EVP_DecryptFinal` for decryption. In any case, do what the OpenSSL team tells you to do. – jww Nov 17 '16 at 02:13

0 Answers0