How to free allocated memory in C++ DLL

Question

i've got following code to encrypt a string in a C++ DLL

EXPORT WCHAR* EncryptString(WCHAR* stringToEncrypt) {
    aes_context ctx;

    WCHAR* in = stringToEncrypt;
    WCHAR* out;
    WCHAR* key = L"TestKey";

    BYTE* buffEnc = (BYTE*)malloc(16);
    BYTE* keyBuffEnc = (BYTE*)malloc(32);

    memset(buffEnc, 0, 16);
    memset(keyBuffEnc, 0, 32);

    memcpy(buffEnc, in, wcslen(in) * 2);
    memcpy(keyBuffEnc, key, wcslen(key) * 2);
    aes_set_key(&ctx, keyBuffEnc, 256);

    aes_encrypt(&ctx, buffEnc, buffEnc);
    out = (WCHAR*)buffEnc;

    // free(buffEnc);   
    // free(keyBuffEnc);

    return out;
}

My problem is that i can not free the allocated memory because otherwise the result is broken. I wonder how can i free the used memory without losing the result? Have i to change the type of return value?

Thanks in advance for your help. Greets Heinz

Answer 1

This is indeed a problematic situation - you return a pointer to allocated memory and it's unclear who should free the memory. You have the following options:

1. tell the caller free the memory using free() - this will only work if they use the same heap which is hard to guarantee. This is very unreliable and not really recommended.
2. introduce a memory management interface (such as freeEncrypted() function that is implemented in your library) and tell the caller use it - then memory will be returned to the right heap.
3. use something well known like CoTaskMemAlloc() for allocation and tell the caller to use the matching function such as CoTaskMemFree() for freeing memory. This is similar to point 2, just uses a well known common memory manager.
4. change the interface such that it accepts pointer to already allocated data and its size so that the caller both allocates and frees the memory.

Answer 2

On Windows, the memory manager (which among other things keeps track of the allocated and free memory in your process) works in the C runtime library. This means that if you have two modules (say: your actual executable and a DLL, or two DLLs) and you want to allow one module to allocate some memory and the other one should own it (i.e. be responsible for freeing it or passing the maintainership on) thre are basically three options:

1. You let the caller allocate a piece of memory and pass a pointer to that to the callee. In your example, that would boil down to the caller allocating a (hopefully sufficiently large) buffer and then passing a pointer to that to the EncryptString function. The advantage of this approach is that the caller can choose to just allocate a piece of memory on the stack and then pass a pointer to that, something like

   WCHAR buf[1024];
   EncryptString( "Hello", buf ); // Won't compile, "Hello" is a const string
   

   The disadvantage is that the caller has to figure out an appropriate buffer size first. You could just impose a maximum limit and document that, or you could have a dedicated function exposed which computes the required size, or you could say that if NULL is passed as the output buffer, the function returns the number of required characters. The latter is commonly used by the Windows API.

2. You let the callee allocate the memory (as your function does right now) using e.g. malloc or new but then dictate that the caller has to use a special function to release the memory again, something like FreeEncryptedString(char *s). The idea is that this deallocation function is also exposed by your DLL and it just calls the appropriate deallocation function (i.e. free or delete or delete[] or the like), so that both allocation and deallocation happen within the same module.

3. You ensure that both modules link against the same C runtime library dynamically, i.e. there is just one copy of the C runtime DLL in the process and both of your modules use it. In that case, you could just use malloc and free (or new and delete etc.) as you want to. The advantage of this is that it's very straightforward, the disadvantage is that it means you impose requirements on how your modules are built (Which may not be possible if you work on a program which loads plugins written by other people or so - those plugins may chose to link against the C runtime statically).