OpenSSL EVP API: How to decrypt the encrypted file using a symmetric key file

Question

Hi I'm working on C on Linux.

I have a query related to symmetric key decryption.

I have generated an symmetric key using the below command.

openssl rand base64 512 > sym.key

Using this key (sym.key) I have encrypted a file with below command.

openssl enc -aes-256-cbc -in temp.txt -out temp.enc -kfile sym.key

It has generated an encrypted file temp.enc.

Now, I have to use the same key (sym.key) with EVP Decrypt API's and have to decrypt this encrypted file.

Could any one suggest me a better approach for this.

Here is the code

unsigned char* decode (unsigned char *key, int len)
{

   BIO *b64, *bmem;

   char *buffer = (char *)malloc(len);
   memset (buffer, 0, len);

   b64 = BIO_new(BIO_f_base64());
   bmem = BIO_new_mem_buf(key, len);
   bmem = BIO_push(b64, bmem);

   BIO_read(bmem, buffer, len);

   BIO_free_all(bmem);

    return buffer;
}

void decrypt(char *file_name, char *key_file)
{   

    unsigned char *inbuff = NULL, *outbuff = NULL, *ckey = NULL;
    char *buff = NULL;
    unsigned int flen = 0, outlen2 = 0, outlen1 = 0, klen = 0;

    FILE *fp = NULL, *kfp = NULL;
        unsigned char iv[16] = {};

    fp = fopen (file_name, "r");

    if (NULL == fp)
    {
        printf ("Cannot open file : %s\n", file_name);
        exit(1);
    }

    fseek (fp, 0, SEEK_END);
    flen = ftell (fp);
    rewind (fp);

    kfp = fopen (key_file, "r");

    if (NULL == kfp)
    {
        printf ("Cannot open file : %s\n", key_file);
        exit(1);
    }

    fseek (kfp, 0, SEEK_END);
    klen = ftell (kfp);
    rewind (kfp);

    inbuff = (unsigned char *)malloc(flen);
    outbuff = (unsigned char *)malloc(flen * 2);
    ckey = (unsigned char *)malloc(klen);
    buff = (char *)malloc(klen);

    fread (inbuff, sizeof(char), flen, fp);
    fread (buff, sizeof(char), klen, kfp);

    ckey = decode(buff, klen);

    EVP_CIPHER_CTX ctx;

#if 1
    if (! EVP_DecryptInit (&ctx, EVP_aes_256_cbc(), ckey, iv))
    {
        ERR_print_errors_fp(stderr);
        EVP_CIPHER_CTX_cleanup(&ctx);
        printf ("Error in Init\n");
        exit(1);
    }

    if (! EVP_DecryptUpdate (&ctx, outbuff, &outlen1, inbuff, flen))
    {
        ERR_print_errors_fp(stderr);
        EVP_CIPHER_CTX_cleanup(&ctx);
        printf ("Error in Init\n");
        exit(1);
    }

    if (! EVP_DecryptFinal (&ctx, outbuff + outlen1, &outlen2))
    {
        ERR_print_errors_fp(stderr);
        EVP_CIPHER_CTX_cleanup(&ctx);
        printf ("Error in Init\n");
        exit(1);
    }

    EVP_CIPHER_CTX_cleanup(&ctx);
#endif

    free (inbuff);
    free (outbuff);
    free (ckey);
    fclose (fp);
    fclose (kfp);

    printf ("Outbuff:\n %s\n", outbuff);

}

Thank you.

Answer 1

Using this key (sym.key) I have encrypted a file with below command.
openssl enc -aes-256-cbc -in temp.txt -out temp.enc -kfile sym.key

Not quite. openssl enc -kfile reads the first line, and only the first line, of the file and uses it as the password, which is not the same thing as the key. enc has three options to enter a password, after which it runs the password with random salt through a key derivation process to produce the actual key and IV (for ciphers that use an IV, and AES-CBC does). It then writes a header containing the salt, followed by the ciphertext, to the output file. This is called Password Based Encryption (PBE) and sometimes a Password Based Key Derivation Function (PBKDF).

Depending on what you actually want to do there are two approaches.

To decrypt the file you have

Read the first line from sym.key (excluding the line terminator) as characters, NOT base64, and feed it with the salt from temp.enc through EVP_BytesToKey something like this:

FILE * pwfile = fopen (key_file, "r"); 
if(!pwfile) error_handling 
char pwline [70]; 
fgets (pwline, sizeof pwline, pwfile);
int pwlen = strlen (pwline); 
if(pwlen==0 || pwline[pwlen-1]!='\n') error_handling
pwline[--pwlen] = '\0';

// Open file_name and read into inbuff for flen as you have now 
// Optionally confirm the first 8 bytes are "Salted__"
// If on Windows must fopen with mode "rb" to get all bytes correctly;
// on Unix it is clearer to specify this but not actually needed
// because on Unix binary files and text files are treated the same

unsigned char key [256/8]; // AES-256 key is 32 bytes
unsigned char iv [128/8]; // AES IV is 16 bytes (regardless of key)
EVP_BytesToKey(EVP_aes_256_cbc(), EVP_md5(), inbuff+8, /* the salt! */
    (unsigned char*)pwline, pwlen, 1, key, iv);

// Now continue as you have with EVP_Encrypt{Init,Update,Final}
// using key,iv except use buffer inbuff+16 for length flen-16 .
// (And do something with the output, which you don't now!)

To create the file you apparently wanted

To use enc to create a file encrypted with a direct key, you must pass it with the -K option (uppercase K) on the commandline in hex. However, it's a nuisance to handle hex in your C program, so I would handle it on the encrypt side something like:

openssl rand 32 >sym.key # AES-256 key must be exactly 32 bytes, not more
openssl enc -aes-256-cbc -in temp.txt -out temp.enc \
  -K $(od -An -tx1 sym.key | sed 's/ //g') -iv 00000000000000000000000000000000

then in your C program read sym.key (as binary on Windows at least) and use it as-is, with an IV of 16 0's as you have now.

Alternatively, have rand write hex and use that as-is

openssl rand -hex 32 >sym.key
openssl enc -aes-256-cbc -in temp.txt -out temp.enc -K $(cat sym.key) -iv (same)

then in your C program read sym.key (as a text line) and convert it from 64 chars of hex to unsigned char key [32].

Other points

If you did need to decode a base64 file, you don't need to read it into memory first then push a b64BIO on a memBIO; you can push a b64BIO on a fileBIO that reads the file.

You don't need to allocate flen*2 for outbuff. Decrypted plaintext will never be longer than the ciphertext. (Which for the salted-PBE form is actually flen-16 as above.) It is encryption that can expand the data, and then only one block (for AES, 16 bytes).

It's good practice to check malloc didn't return NULL before using it. This rarely happens on modern systems, but it can occur if even simple code like this is called from a larger program and some other part of the program has a bug that exhausts memory, or perhaps a vulnerability exploited by a denial-of-service attack.

If you want to support large files that don't fit in the memory available, or might not, iteratively read chunks, feed each into DecryptUpdate, write out the results (which will lag about one block), and at EOF call DecryptFinal and output any last partial block.