Downloading an .exe file in binary format to a buffer in C

Question

I'm trying to download an .exe file from my webserver for licensing purpose. For that I need to :

Download executable file from webserver
Receive that file in binary format ( 0x00/x00 )
I need to verify that the download succeeded using IMAGE_DOS_HEADER->e_magic

This is not very difficult in theory and was easily done with high level language but I struggle doing it in C

I already know how to download a file ( .hmtl, .txt, ... ) from my webserver using the WinHttp library ( WinHttpOpen, WinHttpConnect, WinHttpOpenRequest, WinHttpSetOption, WinHttpSendRequest, WinHttpReceiveResponse, WinHttpQueryDataAvailable, WinHttpReadData ) without any issue.

The code I use look like that :

        DWORD outBufSize = 0;
        LPSTR lpOutBuffer;
        DWORD bytesDownloaded = 0;
        do
        {
            bQueryData = WinHttpQueryDataAvailable(hOpenRequest, &outBufSize);
            if (bQueryData)
                printf("WinHttpQueryDataAvailable Success | Bytes available : %d\n", outBufSize);
            else
                printf("Error using WinHttpQueryDataAvailable : %d\n", GetLastError());


            lpOutBuffer = malloc(outBufSize + 1);
            if (!lpOutBuffer)
            {
                printf("out of memory for lpOutBuffer\n");
                outBufSize = 0;
                free(lpOutBuffer);
            }
            else
            {
                bReadData = WinHttpReadData(hOpenRequest, (LPVOID)lpOutBuffer, outBufSize, &bytesDownloaded);
                if (bReadData)
                {
                    printf("WinHttpReadData Success | Bytes downloaded : %d\n", bytesDownloaded);
                    printf("0x%2x", lpOutBuffer);
                    
                }
                else
                    printf("Error using WinHttpReadData : %d\n", GetLastError());
            }
        } while (outBufSize > 0);
        free(lpOutBuffer);

When I try it with an executable file i get this response

WinHttpQueryDataAvailable Success | Bytes available : 3742
WinHttpReadData Success | Bytes downloaded : 3742

0x85c070
WinHttpQueryDataAvailable Success | Bytes available : 866
WinHttpReadData Success | Bytes downloaded : 866

0x8e4288
WinHttpQueryDataAvailable Success | Bytes available : 0
WinHttpReadData Success | Bytes downloaded : 0

0x8b3820

The output I would expect is something like :

0x45, 0x50, 0x00, 0x00, 0x00, ...

It looks like you're just printing the value of the pointer `lpOutBuffer`. To print hex bytes you'll need to start at that pointer and print bytes until you reach a null terminator or you reach the size `outBufSize` whichever is relevant. — TomServo, Dec 07 '22 at 18:16
The line `printf("0x%2x", lpOutBuffer);` will print the value of the pointer returned by `malloc`, which is the address of the memory buffer. You probably want to print the data at that address instead of the address itself. Therefore, I recommend that you use the following loop instead: `for ( DWORD i = 0; i < bytesDownloaded; i++ ) printf( "0x%02x ", lpOutBuffer[i] );` — Andreas Wenzel, Dec 07 '22 at 18:21
@AndreasWenzel, it's pretty much working well but for a reason that i ignore i get a lot of "0xfffffff" value. Ex : 0x4d 0x5a 0xffffff90 0x00. — RenardoSharp, Dec 07 '22 at 18:43
@RenardoSharp: Ah, sorry, that is my fault. Change `printf( "0x%02x ", lpOutBuffer[i] );` to `printf( "0x%02x ", (unsigned char)lpOutBuffer[i] );`. That way, you will no longer be passing negative values to `printf`. — Andreas Wenzel, Dec 07 '22 at 18:47
@AndreasWenzel Last question, what should I do to store this buffer dynamically so I can use if after in another function in the same program, do I need to create a new LPSTR/BYTE buffer and memcpy from lpOutBuffer to this newly created buffer ? if so how can I do it since my buffer size is reset to 0 every loop or should I simply do everything from inside the for loop ? — RenardoSharp, Dec 07 '22 at 21:21
@RenardoSharp: One thing you could do is to increase the size of the buffer as necessary, using the function `realloc`. However, beware that this may cause the entire buffer to be copied to the new memory location every time, which will give you a [time complexity](https://en.wikipedia.org/wiki/Time_complexity) of `O(n^2)` . In other words, this can be a very inefficient algorithm with large amounts of data. A more efficient algorithm would grow the buffer exponentially, for example by doubling the size of the buffer every time you call `realloc`, which gives you a time complexity of `O(n)`. — Andreas Wenzel, Dec 07 '22 at 23:35
@RenardoSharp: You may want to take a look at [this answer of mine](https://stackoverflow.com/a/73410595/12149471) to another question, which reads a file into memory, using an exponentially growing buffer. This is essentially the same thing as what you want to do. — Andreas Wenzel, Dec 07 '22 at 23:41
@AndreasWenzel I'm afraid that using this method I will have free space on the dynamically created buffer, the final buffer MUST have the exact same number of bytes that the .exe file have, it cannot have more bytes than that otherwise it will somehow crash my application, as written on my question I have to do things file the file structure, if the file itself is corrupted it won't work, that's why I've though about using memcpy and creating a bufPtr that will be increased by the "bytesDownloaded" variable. ex : memcpy(newBuffer+ bufPtr, lpOutBuffer, bytesDownloaded), what do you think ? — RenardoSharp, Dec 07 '22 at 23:56
@RenardoSharp: You can have an additional counter variable which keeps track of the total number of bytes downloaded, so that you always know how many bytes in the buffer are valid. That way, it should not be a problem if the buffer is larger than necessary. This is what I do in my answer linked above. Also, after you have finished reading the entire file, you can use `realloc` to shrink the buffer to the number of bytes actually downloaded, so that no memory is wasted. Therefore, I do not see a problem. — Andreas Wenzel, Dec 08 '22 at 00:02
@AndreasWenzel You are right, in fact I can shrink the buffer at the end of the do/while loop, I didn't see it like that in first, thanks for your help, have a good day/night :) — RenardoSharp, Dec 08 '22 at 00:05
@RenardoSharp: The two disadvantages of the algorithm I described is that (1) it may temporarily allocate up to double the memory that is actually required, which may cause a failure if insufficient memory is available, and (2) the data may have to be copied multiple times by `realloc`, and the total number of bytes copied by `realloc` could be up to twice the total amount of data. Normally, these two disadvantages should not be a problem. However, it is possible to get rid of these disadvantages, by having the file content in multiple blocks of memory (which I do not recommend). — Andreas Wenzel, Dec 08 '22 at 00:15

RenardoSharp · Answer 1 · 2022-12-07T21:14:09.510

With the help of comment we managed to find the solution, working code :

    DWORD outBufSize = 0;
    LPSTR lpOutBuffer;
    DWORD bytesDownloaded = 0;
    do
    {
        bQueryData = WinHttpQueryDataAvailable(hOpenRequest, &outBufSize);
        if (bQueryData)
            printf("WinHttpQueryDataAvailable Success | Bytes available : %d\n", outBufSize);
        else
            printf("Error using WinHttpQueryDataAvailable : %d\n", GetLastError());


        lpOutBuffer = malloc(outBufSize + 1);
        if (!lpOutBuffer)
        {
            printf("out of memory for lpOutBuffer\n");
            outBufSize = 0;
            free(lpOutBuffer);
        }
        else
        {
            bReadData = WinHttpReadData(hOpenRequest, (LPVOID)lpOutBuffer, outBufSize, &bytesDownloaded);
            if (bReadData)
            {
                printf("WinHttpReadData Success | Bytes downloaded : %d\n", bytesDownloaded);
                for(DWORD i = 0; i < bytesDownloaded; i++)
                    printf("0x%02x ", (unsigned char)lpOutBuffer[i]);
                
            }
            else
                printf("Error using WinHttpReadData : %d\n", GetLastError());
        }
    } while (outBufSize > 0);
    free(lpOutBuffer);

Downloading an .exe file in binary format to a buffer in C

1 Answers1