What do the three stars signify in this code for hooking the Direct3D EndScene call?

Question

I'm doing some tutorials about hooking endScene and I don't understand why there are three *** instead of one. What are the second and the third ones representing:

memcpy(pTable, *(void***)(pDummyDevice), size);

I can't understand why do I put three '*' instead of just a one if someone can explain to me why there are three I would very appreciate it

    bool GetD3D9Device(void** pTable, size_t size)
{
    if (!pTable)
        return false;

    IDirect3D9* pD3D = Direct3DCreate9(D3D_SDK_VERSION);

    if (!pD3D) {
        return false;
    }
    IDirect3DDevice9* pDummyDevice = nullptr;

    D3DPRESENT_PARAMETERS d3dpp = {};
    d3dpp.Windowed = false;
    d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;
    d3dpp.hDeviceWindow = GetProcessWindow();

    HRESULT dummyDevCreated = pD3D->CreateDevice(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, d3dpp.hDeviceWindow,D3DCREATE_SOFTWARE_VERTEXPROCESSING,&d3dpp,&pDummyDevice);

    if (dummyDevCreated != S_OK) {
        d3dpp.Windowed = !d3dpp.Windowed;
        HRESULT dummyDevCreated = pD3D->CreateDevice(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, d3dpp.hDeviceWindow, D3DCREATE_SOFTWARE_VERTEXPROCESSING, &d3dpp, &pDummyDevice);

        if (dummyDevCreated != S_OK)
        {
            pD3D->Release();
            return false;
        }
    }

    memcpy(pTable, *(void***)(pDummyDevice), size);
    pDummyDevice->Release();
    pD3D->Release();
}

  void* d3d9Device[119]; // this is the var

You already know this: https://stackoverflow.com/questions/1461432/what-is-array-to-pointer-decay?r=Saves_AllUserSaves ? — πάντα ῥεῖ, Mar 24 '23 at 12:37
I don't think this is a terrible question, though it doesn't seem answerable here. (And pointer decay isn't relevant.) It converts pDummyDevice, o type IDirect3DDevice9*, to void***, then finds what it points to, which becomes void**. memcpy expects void*. This could be a typo that memcpy just doesn't care about. All it's going to do is pass pDummyDevice as an address anyway, and it couldn't care less what it's the address of. So my suggestion is you don't care either. :) — Topological Sort, Mar 24 '23 at 13:21
@TopologicalSort Yes, the point of this snippet is to dereference `pDummyDevice` such that the `src` argument to `memcpy` is the address of the (static) vtable. — Botje, Mar 24 '23 at 13:55

Botje · Accepted Answer · 2023-04-07T13:28:25.253

The short version of the story is that this code attempts to extract the direct function pointer table of a IDirect3DDevice9. Presumably in order to hook specific function calls by either replacing the vtable entry with a hooked variant or by placing a redirection at the start of the code of individual functions.

Conceptually, you can think of a IDirect3DDevice9 object as having the following structure:

struct IDirect3DDevice9 {
  IDirect3DDevice9_vtbl *vtable;
  // private data members of this object
};

struct IDirect3DDevice9_vtbl {
  IUnknown_vtbl * unk;
  FNPTR TestCooperativeLevel;
  FNPTR GetAvailableTextureMem;
  // about 100 more function pointers, including `Render`, `Present`, `BeginScene`, `EndScene`, etc.
  FNPTR DeletePatch;
  FNPTR CreateQuery;
}

Somewhere in the code of the game you are hooking, there will be a call to dev->EndScene(), which translates to the following instructions:

Load the vtable from dev: IDirect3DDevice9_vtbl& vtbl = *dev->vtable; Since vtable is the first member, this compiles to the the same instruction as the expression *(void *)dev.
Grab the function from the vtable: FNPTR IDirect3DDevice9_vtbl_EndScene = vtbl.EndScene. Endscene is at offset 40 of the vtable, so you could also say ((FNPTR *)&vtbl)[40] or even *((FNPTR *)&vtbl + 40).
Invoke the function with this bound to dev.

If you ignore the types for a second, you have a pointer (one star) to a vtable, which contains an array of function pointers (two stars). Because all these function pointers have a different type, your code sample has chosen to treat them as void pointers (there's your third star).

All of this hinges on an implementation detail of the C++ compiler being used, namely that every object with virtual functions has a pointer to a vtable at the start, that all instances of a class share the same vtable, and that all functions are laid out sequentially inside the vtable. Of course the Direct3D API has changed over time, but this was always done by appending new functions to the end of the vtable, which makes existing code keep working.

What do the three stars signify in this code for hooking the Direct3D EndScene call?

1 Answers1