DirectX9 CreateDevice() returns D3DERR_INVALIDCALL in injected DLL for VMT hooking

Question

I want to modify a DirectX-Application's behavior (namely I want to implement a program similar to the Statman-Application by OrfeasZ [https://github.com/OrfeasZ/Statman/releases] as Onscreen-Info for Hitman 2) by injecting code (as DLL) into it and hooking the DirectX DeviceInterface VMT.

Since there are very limited resources on how to do this for DirectX11-Applications, I first wanted to learn how to do this in DX9 by creating a program that gets the DeviceInterface pointer of any DirectX9-Application. I wrote this code in the DllMain() function of my DLL (which is almost a 1:1 copy/paste of the third answer to this thread Hooking DirectX EndScene from an injected DLL):

HMODULE hDLL = GetModuleHandleA("d3d9");
LPDIRECT3D9(__stdcall*pDirect3DCreate9)(UINT) = (LPDIRECT3D9(__stdcall*)(UINT))GetProcAddress(hDLL, "Direct3DCreate9");

LPDIRECT3D9 pD3D = pDirect3DCreate9(D3D_SDK_VERSION);

D3DDISPLAYMODE d3ddm;
HRESULT hRes = pD3D->GetAdapterDisplayMode(D3DADAPTER_DEFAULT, &d3ddm);
D3DPRESENT_PARAMETERS d3dpp;
ZeroMemory(&d3dpp, sizeof(d3dpp));
d3dpp.Windowed = true;
d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;
d3dpp.BackBufferFormat = d3ddm.Format;

//WNDPROC TempWndProc;
WNDCLASSEX wc = { sizeof(WNDCLASSEX),CS_CLASSDC, WndProc,0L,0L,GetModuleHandle(NULL),NULL,NULL,NULL,NULL,TEXT("1"),NULL };
RegisterClassEx(&wc);
HWND hWnd = CreateWindow(TEXT("1"), NULL, WS_OVERLAPPEDWINDOW, 100, 100, 300, 300, GetDesktopWindow(), NULL, wc.hInstance, NULL);

ShowWindow(hWnd, SW_SHOW);

IDirect3DDevice9 * ppReturnedDeviceInterface;
hRes = pD3D->CreateDevice(
    D3DADAPTER_DEFAULT,
    D3DDEVTYPE_HAL,
    hWnd,
    D3DCREATE_SOFTWARE_VERTEXPROCESSING,
    &d3dpp, &ppReturnedDeviceInterface);

pD3D->Release();
DestroyWindow(hWnd);

if (pD3D == NULL) {
    //printf ("WARNING: D3D FAILED");
    return false;
}
unsigned long* pInterface = (unsigned long*)*((unsigned long*)ppReturnedDeviceInterface);

When I inject the DLL into a DirectX9-Application (I've tried this with Civilization V and Total War: Shogun 2), it opens a window, so it actually is able to get the Direct3DCreate9 function from the d3d9.dll within the game, but pD3D->CreateDevice() always returns `D3DERR_INVALIDCALL. I don't really get what could be the reason for this, especially since the rest of this program works flawlessly. Does anybody have any idea what is missing/wrong?

score 0 · Answer 1 · answered Mar 18 '19 at 08:21

D3DERR_INVALIDCALL

The method call is invalid. For example, a method's parameter may not be a valid pointer.

Based on the error information this issue may caused by invalid parameter of IDirect3D9::CreateDevice method. You need initialize the pointer:

   IDirect3DDevice9 *pReturnedDeviceInterface = NULL;

Also check if hWnd is a valid window handle and d3ddm.Format etc.

DirectX9 CreateDevice() returns D3DERR_INVALIDCALL in injected DLL for VMT hooking

1 Answers1