A FIPS validated OpenSSL library must load libeay32.dll at a fixed address. This is indicated, for example, at: Fixed address is occupied in .NET
Using techniques described at: IIS7 App Pool can't load library with Fixed Base Address, and How do you disable ASLR (address space layout randomization) on Windows 7 x64?, and How to run a PE image without linking kernel32.dll and ntdll.dll, I've forced the loading of the FIPS DLL as early as I am able.
For a 64-bit process, from a ListDLLs output, I see a load order of:
E:\my\my64.exe
C:\Windows\SYSTEM32\ntdll.dll
C:\Windows\system32\kernel32.dll
C:\Windows\system32\KERNELBASE.dll
E:\my\LIBEAY_FIPS32.dll
...
For a 32-bit process, there is some extra "wow" nonsense, namely:
E:\my\my32.exe
C:\Windows\SYSTEM32\ntdll.dll
C:\Windows\SYSTEM32\wow64.dll
C:\Windows\SYSTEM32\wow64win.dll
C:\Windows\SYSTEM32\wow64cpu.dll
C:\Windows\SysWOW64\ntdll.dll
C:\Windows\syswow64\kernel32.dll
C:\Windows\syswow64\KERNELBASE.dll
E:\my\LIBEAY_FIPS32.dll
...
Now, this is a huge improvement from what I started with, and the chances of getting a "0xc0000018 failed to load at FIXED address error" is now much lower than when LIBEAY_FIPS32.dll was around the 25th DLL loaded into the process.
However, given that these system DLLs are relocatable and presumably subject to ASLR, I expect load failures are still possible.
Though I don't want to switch off Windows ASLR, I would like to somehow tell ASLR to "please avoid the following FIXED address range because I really need to load a FIXED DLL at that address". Is there any way to do that?
Or some way to control the load address of one or more of: ntdll.dll, kernel32.dll, kernelbase.dll?
Failing that, is there any way to force the FIPS openssl dll to load into a process before one or more of: ntdll.dll, kernel32.dll, kernelbase.dll?
