How do I reserve memory regions before Windows maps my program's DLLs?

Question

My Windows program needs to use very specific regions of memory. Unfortunately, Windows loads quite a few DLLs in memory and because of ASLR, their locations are not predictable, so they could end up being mapped into regions that my program needs to use. On Linux, Wine solves this problem by using a preloader application which reserves memory regions and then manually loads and executes the actual image and dynamic linker. I assume that specific method is not possible on Windows, but is there another way to get reserved regions of memory that are guaranteed to not be used by DLLs or the process heap?

If it helps, the memory regions are fixed and known at compile time. Also, I'm aware that ASLR can be disabled system-wide in the registry or per-process using the Enhanced Mitigation Experience Toolkit, but I don't want to require my users to do that.

Answer 1

I think I finally got it using a method similar to what dxiv suggested in the comments. Instead of using a dummy DLL, I build a basic executable that loads at the beginning of my reserved region using the /FIXED and /BASE compiler flags. The code for the executable contains an uninitialized array that ensures the image covers the needed addresses in memory, but doesn't take up any extra space in the file:

unsigned char Reserved[4194304]; // 4MB

At runtime, the executable copies itself to a new location in memory and updates a couple of fields in the Process Environment Block to point to it. Without updating the fields, calling certain functions like FormatMessage would cause a crash.

#include <intrin.h>
#include <windows.h>
#include <winternl.h>

#pragma intrinsic(__movsb)

void Relocate() {
    void *Base, *NewBase;
    ULONG SizeOfImage;
    PEB *Peb;
    LIST_ENTRY *ModuleList, *NextEntry;

    /* Get info about the PE image. */
    Base = GetModuleHandleW(NULL);
    SizeOfImage = ((IMAGE_NT_HEADERS *)(((ULONG_PTR)Base) +
        ((IMAGE_DOS_HEADER *)Base)->e_lfanew))->OptionalHeader.SizeOfImage;

    /* Allocate memory to hold a copy of the PE image. */
    NewBase = VirtualAlloc(NULL, SizeOfImage, MEM_COMMIT, PAGE_READWRITE);
    if (!NewBase) {
        ExitProcess(GetLastError());
    }

    /* Copy the PE image to the new location using __movsb since we don't have
       a C library. */
    __movsb(NewBase, Base, SizeOfImage);

    /* Locate the Process Environment Block. */
    Peb = (PEB *)__readfsdword(0x30);

    /* Update the ImageBaseAddress field of the PEB. */
    *((PVOID *)((ULONG_PTR)Peb + 0x08)) = NewBase;

    /* Update the base address in the PEB's loader data table. */
    ModuleList = &Peb->Ldr->InMemoryOrderModuleList;
    NextEntry = ModuleList->Flink;
    while (NextEntry != ModuleList) {
        LDR_DATA_TABLE_ENTRY *LdrEntry = CONTAINING_RECORD(
            NextEntry, LDR_DATA_TABLE_ENTRY, InMemoryOrderLinks);
        if (LdrEntry->DllBase == Base) {
            LdrEntry->DllBase = NewBase;
            break;
        }
        NextEntry = NextEntry->Flink;
    }
}

I built the executable with /NODEFAULTLIB just to reduce its size and the number of DLLs loaded at runtime, hence the use of the __movsb intrinsic. You could probably get away with linking to MSVCRT if you wanted to and then replace __movsb with memcpy. You can also import memcpy from ntdll.dll or write your own.

Once the executable is moved out of the way, I call a function in a DLL that contains the rest of my code. The DLL uses UnmapViewOfFile to get rid of the original PE image, which gives me a nice 4MB+ chunk of memory to work with, guaranteed not to contain mapped files, thread stacks, or heaps.

A few things to keep in mind with this technique:

1. This is a huge hack. I felt dirty writing it and it very well could fall apart in future versions of Windows. I also haven't tested this on anything other than Windows 7. This code works on Windows 7 and Windows 10, at least.
2. Since the executable is built with /FIXED /BASE, its code is not position-independent and you can't just jump to the relocated executable.
3. If the DLL function that calls UnmapViewOfFile returns, the program will crash because the code section we called from doesn't exist anymore. I use ExitProcess to ensure the function never returns.
4. Some sections in the relocated PE image like those containing code can be released using VirtualFree to free up some physical memory.
5. My code doesn't bother re-sorting the loader data table entries. It seems to work fine that way, but it could break if something were to depend on the entries being ordered by image address.
6. Some anti-virus programs might get suspicious about this stuff. Microsoft Security Essentials didn't complain, at least.
7. In hindsight, dxiv's dummy DLL method may have been easier, because I wouldn't need to mess with the PEB. But I stuck with this technique because the executable is more likely to be loaded at its desired base address. The dummy DLL method didn't work for me. DLLs are loaded by Ntdll after Windows has already reserved regions of memory that I need.