WinAPI - CryptDecrypt() not working properly in AES 256

Question

I used to work with crypto++ in Visual Studio before, but now I want to use of wincrypt.h API functions to encrypt a string with AES 256 with an IV (cbc mode).

I did bellow steps but I'm confused about CryptEncrypt() and CryptDecrypt() functions, because It seems they don't work properly :

CryptAcquireContextA defined to create a CSP :

// create a cryptographic service provider (CSP)
CryptAcquireContextA(&hProv, NULL, MS_ENH_RSA_AES_PROV_A, PROV_RSA_AES, CRYPT_VERIFYCONTEXT)

For set key, I'm using of this way (import key):

CryptImportKey(hProv, (BYTE*)&AESBlob, sizeof(AES256KEYBLOB), NULL, CRYPT_EXPORTABLE, &hKey)

IV, Key, Plaintext sizes are :

#define     DEFAULT_AES_KEY_SIZE    32
#define     DEFAULT_IV_SIZE         16
#define     BUFFER_FOR_PLAINTEXT    32

This is my whole code :

// handles for csp and key
HCRYPTPROV hProv = NULL;
HCRYPTPROV hKey = NULL;
BYTE szKey[DEFAULT_AES_KEY_SIZE + 1] = {0};
BYTE szIV[DEFAULT_IV_SIZE + 1] = {0};
// plain bytes
BYTE szPlainText[BUFFER_FOR_PLAINTEXT + 1] = {0};
DWORD dwPlainSize = 0;

// initalize key and plaintext
StrCpyA((LPSTR)szKey, "00112233445566778899001122334455");
StrCpyA((LPSTR)szIV, "4455667788990011");
StrCpyA((LPSTR)szPlainText, "abcdefghijklmnopqrstuvwxyzabcdef");

// blob data for CryptImportKey() function (include key and version and so on...)
struct AES256KEYBLOB
{
    AES256KEYBLOB() { StrCpyA((LPSTR)szBytes, 0); }
    BLOBHEADER bhHdr;
    DWORD dwKeySize;
    BYTE szBytes[DEFAULT_AES_KEY_SIZE + 1];
} AESBlob;

AESBlob.bhHdr.bType = PLAINTEXTKEYBLOB;
AESBlob.bhHdr.bVersion = CUR_BLOB_VERSION;
AESBlob.bhHdr.reserved = 0;
AESBlob.bhHdr.aiKeyAlg = CALG_AES_256;
AESBlob.dwKeySize = DEFAULT_AES_KEY_SIZE;
StrCpyA((LPSTR)AESBlob.szBytes, (LPCSTR)szKey);

// create a cryptographic service provider (CSP)
if(CryptAcquireContextA(&hProv, NULL, MS_ENH_RSA_AES_PROV_A, PROV_RSA_AES, CRYPT_VERIFYCONTEXT))
{
    if(CryptImportKey(hProv, (BYTE*)&AESBlob, sizeof(AES256KEYBLOB), NULL, CRYPT_EXPORTABLE, &hKey))
    {
        if(CryptSetKeyParam(hKey, KP_IV, szIV, 0))
        {
            if(CryptEncrypt(hKey, NULL, TRUE, 0, szPlainText, &dwPlainSize, lstrlenA((LPCSTR)szPlainText) + 1))
            {
                printf("\nEncrypted data : %s\nSize : %d\n", (LPCSTR)szPlainText, dwPlainSize);

                if(CryptDecrypt(hKey, NULL, TRUE, 0, szPlainText, &dwPlainSize)) {
                    printf("\nDecrypted data : %s\nSize : %d\n", (LPCSTR)szPlainText, dwPlainSize);
                }
                else
                    printf("failed to decrypt!");
            }
            else
                printf("failed to encrypt");
        }
    }
}

It just encrypt half section of plaintext And decrypting not done! even By changing just szPlainText value, it always give me bellow output (It means that CryptEncrypt() and CryptDecrypt() does not working as expected!) :

Encrypted data : U╡π7ÑL|FΩ$}├rUqrstuvwxyzabcdef
Size : 16

Decrypted data : U╡π7ÑL|FΩ$}├rUqrstuvwxyzabcdef
Size : 0

I think you need to copy the key for the decrypt step - encryption morphs the key. — 500 - Internal Server Error, Aug 14 '18 at 11:07
@500-InternalServerError when i print the key before decrypting, it is correct with out any chenges — BattleTested_закалённый в бою, Aug 14 '18 at 11:16
Surely these functions work correctly. Surely the error is in your code. — David Heffernan, Aug 14 '18 at 11:24
@DavidHeffernan I checked every thing. Yes functions working correctly, but i don't know where am i wrong? — BattleTested_закалённый в бою, Aug 14 '18 at 11:28
@500-InternalServerError i checked `hKey`. it has the same value. `if(!hKey) printf("ok");` — BattleTested_закалённый в бою, Aug 14 '18 at 11:41
`hKey` is just a handle to the operating system's internal data structure, which holds a copy of the key that you supplied. This is the key that gets morphed. — 500 - Internal Server Error, Aug 14 '18 at 12:37
@CristiFati this function copy the values to `key` , `iv` and `plaintext` — BattleTested_закалённый в бою, Aug 14 '18 at 13:08
I read the 3 lines of code as well, and based on its name, i figured out what it's supposed to do, but where is its body, to see what it actually does? — CristiFati, Aug 14 '18 at 13:09
As I stated in my previous comment, I noticed where **it's used** (and *BTW*, you **don't have** a `main`), I wanted to see its definition (whether it's a function, a macro, ...). Are you able to provide that? If yes, please include it in the question. — CristiFati, Aug 14 '18 at 13:23
Oh... *shlwapi* was the missing point. I *Google*d the function and didn't see any results (on the 1st page), that's why I thought it was yours. For testing purposes I ussed: `#define StrCpyA strcpy`. — CristiFati, Aug 14 '18 at 13:31
@500-InternalServerError - in general this is true, but here `Final = true` - as result key state reset to intial state — RbMm, Aug 14 '18 at 14:34
you wrong called `CryptEncrypt` - last 2 arguments is invalid. `dwPlainSize` - is inout parameter. on input it must containing size of data - so `lstrlenA((LPCSTR)szPlainText) + 1)` - but you pass 0 here (say that you have 0 size data). and the last parameter - must be size of buffer, not size of data — RbMm, Aug 14 '18 at 14:38
@RbMm you are absolutely right... i didn't attention to that. but i solved my problem, but now i got it what you said to me. any way thank you so much — BattleTested_закалённый в бою, Aug 15 '18 at 03:43

CristiFati · Accepted Answer · 2019-06-04T15:27:51.280

Here's a variant, that I'm running from VStudio 2015.

code.c:

#include <windows.h>
#include <wincrypt.h>
#include <stdio.h>
#include <Shlwapi.h>

#define DEFAULT_AES_KEY_SIZE 32
#define DEFAULT_IV_SIZE 16
#define BUFFER_FOR_PLAINTEXT 32

#define CLEANUP_CRYPT_STUFF(PROV, KEY) \
    CryptDestroyKey(KEY); \
    CryptReleaseContext(PROV, 0)

#define PRINT_FUNC_ERR_AND_RETURN(FUNC) \
    printf("%s (line %d) failed: %d\n", ##FUNC, __LINE__, GetLastError()); \
    return -1


typedef struct AES256KEYBLOB_ {
    BLOBHEADER bhHdr;
    DWORD dwKeySize;
    BYTE szBytes[DEFAULT_AES_KEY_SIZE + 1];
} AES256KEYBLOB;


int main() {
    // handles for csp and key
    HCRYPTPROV hProv = NULL;
    HCRYPTKEY hKey = NULL;
    BYTE szKey[DEFAULT_AES_KEY_SIZE + 1] = { 0 };
    BYTE szIV[DEFAULT_IV_SIZE + 1] = { 0 };
    // plain bytes
    BYTE szPlainText[BUFFER_FOR_PLAINTEXT + 1] = { 0 }, *pBuf = NULL;
    AES256KEYBLOB AESBlob;
    memset(&AESBlob, 0, sizeof(AESBlob));

    // initalize key and plaintext
    StrCpyA((LPSTR)szKey, "00112233445566778899001122334455");
    StrCpyA((LPSTR)szIV, "4455667788990011");
    StrCpyA((LPSTR)szPlainText, "abcdefghijklmnopqrstuvwxyzabcdef");
    DWORD dwPlainSize = lstrlenA((LPCSTR)szPlainText), dwBufSize = dwPlainSize, dwBufSize2 = dwPlainSize;

    // blob data for CryptImportKey() function (include key and version and so on...)
    AESBlob.bhHdr.bType = PLAINTEXTKEYBLOB;
    AESBlob.bhHdr.bVersion = CUR_BLOB_VERSION;
    AESBlob.bhHdr.reserved = 0;
    AESBlob.bhHdr.aiKeyAlg = CALG_AES_256;
    AESBlob.dwKeySize = DEFAULT_AES_KEY_SIZE;
    StrCpyA((LPSTR)AESBlob.szBytes, (LPCSTR)szKey);

    // create a cryptographic service provider (CSP)
    if (!CryptAcquireContextA(&hProv, NULL, MS_ENH_RSA_AES_PROV_A, PROV_RSA_AES, CRYPT_VERIFYCONTEXT)) {
        PRINT_FUNC_ERR_AND_RETURN(CryptAcquireContextA);
    }
    if (!CryptImportKey(hProv, (BYTE*)&AESBlob, sizeof(AES256KEYBLOB), NULL, CRYPT_EXPORTABLE, &hKey)) {
        CryptReleaseContext(hProv, 0);
        PRINT_FUNC_ERR_AND_RETURN(CryptImportKey);
    }
    if (!CryptSetKeyParam(hKey, KP_IV, szIV, 0)) {
        CLEANUP_CRYPT_STUFF(hProv, hKey);
        PRINT_FUNC_ERR_AND_RETURN(CryptSetKeyParam);
    }
    if (CryptEncrypt(hKey, NULL, TRUE, 0, NULL, &dwBufSize, 0)) {
        printf("%d bytes required to hold the encrypted buf\n", dwBufSize);
        if ((pBuf = calloc(dwBufSize, sizeof(BYTE))) == NULL)
        {
            CLEANUP_CRYPT_STUFF(hProv, hKey);
            PRINT_FUNC_ERR_AND_RETURN(calloc);
        }
        StrCpyA(pBuf, szPlainText);
    } else {
        CLEANUP_CRYPT_STUFF(hProv, hKey);
        PRINT_FUNC_ERR_AND_RETURN(CryptEncrypt);
    }
    if (CryptEncrypt(hKey, NULL, TRUE, 0, pBuf, &dwBufSize2, dwBufSize)) {
        printf("\nEncrypted data: [%s]\nSize: %d\n", (LPCSTR)pBuf, dwBufSize2);
        if (CryptDecrypt(hKey, NULL, TRUE, 0, pBuf, &dwBufSize)) {
            printf("\nDecrypted data: [%s]\nSize: %d\n", (LPCSTR)pBuf, dwBufSize);
        } else {
            free(pBuf);
            CLEANUP_CRYPT_STUFF(hProv, hKey);
            PRINT_FUNC_ERR_AND_RETURN(CryptDecrypt);
        }
    } else {
        free(pBuf);
        CLEANUP_CRYPT_STUFF(hProv, hKey);
        PRINT_FUNC_ERR_AND_RETURN(CryptEncrypt);
    }
    free(pBuf);
    CLEANUP_CRYPT_STUFF(hProv, hKey);
    return 0;
}

Notes:

Removed AES256KEYBLOB constructor as I got Access Violation (which is normal when playing with memory "that's not yours"). Replaced the structure initialization with the memset call
The main (logical) error was that the buffer was not large enough to store the encrypted text (combined with a wrong value (0) for dwPlainSize - making the function to misleadingly succeed). According to [MS.Docs]: CryptEncrypt function:

If this parameter contains NULL, this function will calculate the required size for the ciphertext and place that in the value pointed to by the pdwDataLen parameter.

To find out the required size, make an additional call to the function, with pbData set to NULL (this practice is also encountered in other WinAPIs). Then allocate a buffer, fill it with required data, and make the "main" call the function on that buffer...
Added the missing #includes and main
Added code to release the used resources when no longer needed
Refacorings:
- Negated the if conditions, since I don't like so many nesting levels
- Added some convenience macros (CLEANUP_CRYPT_STUFF, PRINT_FUNC_ERR_AND_RETURN), to avoid code duplication
Other minor fixes / improvements
You might want to add a function that prints exactly N bytes from a buffer, as "%s" specifier only stops when "\0" is encountered, and only (dumb) luck prevented the console to be filled with garbage (or even the program to crash) when printfing the buffers
There might be some other aspects related to this functions that I didn't handle (as I'm not an expert in this area), but the goal was just to have smth working

Output:

48 bytes required to hold the encrypted buf

Encrypted data: [<É╙åh∩φ:bOPs  r2w~w╪c╟D╡ï╥V╟neΓßv∩·J8cÅ╥²²²²s]
Size: 48

Decrypted data: [abcdefghijklmnopqrstuvwxyzabcdefΓßv∩·J8cÅ╥²²²²s]
Size: 32

but I didn't know what is this : `CryptEncrypt(hKey, NULL, TRUE, 0, NULL, &dwBufSize, 0)` — BattleTested_закалённый в бою, Aug 14 '18 at 19:14
You're welcome! That's the 2nd bullet: getting the required encrypted buffer size. — CristiFati, Aug 14 '18 at 19:19
Hmmm @cristifati, only a few things wrong with the original code then (!)r — Paul Sanders, Aug 16 '18 at 09:15
@PaulSanders: I don't understand your comment. I listed everything that I had to do in order to have a compilable and working example. Except for the `if` conditions change (to avoid multiple nesting levels) and convenience macros which are refactors (based on personal preferences), everythiong else was required. — CristiFati, Aug 16 '18 at 10:14

score 3 · Answer 2 · answered Aug 14 '18 at 21:40

CristiFati answer is great, And it make me offer use of bellow statement for calculate cipher length :

CryptEncrypt(hKey, NULL, TRUE, 0, NULL, &dwBufSize, 0);

According Microsoft Doc :

If pbData is NULL, no error is returned, and the function stores the size of the encrypted data, in bytes, in the DWORD value pointed to by pdwDataLen) :

BOOL CryptEncrypt(
  HCRYPTKEY  hKey,
  HCRYPTHASH hHash,
  BOOL       Final,
  DWORD      dwFlags,
  BYTE       *pbData,
  DWORD      *pdwDataLen,
  DWORD      dwBufLen
);

My Solution :

But in my code, I just forget to calculate szPlainText size when i give it to CryptEncrypt() :

DWORD dwPlainSize = 0;    // initialized with 0

So "zero-length" has not any mean for bellow function (CryptEncrypt() function always get a plain-text with 0 length) :

CryptEncrypt(hKey, NULL, TRUE, 0, szPlainText, &dwPlainSize, lstrlenA((LPCSTR)szPlainText) + 1)

And I should set its size with bellow statement (My code will be worked just by adding this):

dwPlainSize = lstrlenA((LPCSTR)szPlainText);

Then pass it in right case :

CryptEncrypt(hKey, NULL, TRUE, 0, szPlainText, &dwPlainSize, BUFFER_FOR_PLAINTEXT)

So, output is like bellow :

Encrypted data : <É╙åh∩φ:bOPs  r2w~w╪c╟D╡ï╥V╟neΓßv∩·J8cÅ╥
Size : 48

Decrypted data : abcdefghijklmnopqrstuvwxyzabcdefΓßv∩·J8cÅ╥
Size : 32

Note that you should also add the resource deallocation (`hProv`, `hKey`). — CristiFati, Aug 16 '18 at 10:21
@CristiFati sure i did that before in my code, but i forgot to add it in my question. I know this --> `CryptReleaseContext(hProv, 0)` — BattleTested_закалённый в бою, Aug 16 '18 at 12:01
To print out only the 32 decrypted bytes, try printf("\nDecrypted data: [%.*s]\nSize: %d\n", dwBufSize, (LPCSTR)pBuf, dwBufSize); — vengy, Sep 03 '21 at 20:20

WinAPI - CryptDecrypt() not working properly in AES 256

2 Answers2

Linked