Problem calling DX12 functions in a program compiled with clang in c11

Question

For the sake of learning, I am trying to correctly call the DirectX12 APIs using clang in pure c11. I managed to get everything to compile, but sometimes the vtable pointers seem to get corrupted and change where they point when they shouldn't be.

ID3D12DeviceVtbl* tbl; //tracking the vtable pointer for debugging purposes

D3D_FEATURE_LEVEL featureLevel = D3D_FEATURE_LEVEL_11_0;
if (D3D12CreateDevice(NULL, featureLevel, &IID_ID3D12Device, (void**)&g_pd3dDevice) != S_OK)
{
    return false;
}

{
    tbl = g_pd3dDevice->lpVtbl;

    D3D12_DESCRIPTOR_HEAP_DESC desc ;
    desc.Type = D3D12_DESCRIPTOR_HEAP_TYPE_RTV;
    desc.NumDescriptors = NUM_BACK_BUFFERS;
    desc.Flags = D3D12_DESCRIPTOR_HEAP_FLAG_NONE;
    desc.NodeMask = 1;

    // works fine
    if (g_pd3dDevice->lpVtbl->CreateDescriptorHeap(g_pd3dDevice,&desc, &IID_ID3D12DescriptorHeap, (void**)&g_pd3dRtvDescHeap) != S_OK)
        return false;

    // works fine
    SIZE_T rtvDescriptorSize = g_pd3dDevice->lpVtbl->GetDescriptorHandleIncrementSize(g_pd3dDevice,D3D12_DESCRIPTOR_HEAP_TYPE_RTV);

    // works fine
    D3D12_CPU_DESCRIPTOR_HANDLE rtvHandle = g_pd3dRtvDescHeap->lpVtbl->GetCPUDescriptorHandleForHeapStart(g_pd3dRtvDescHeap);

    // after the line above executes, g_pd3dDevice->lpVtbl now points to somewhere new and invalid
}

{
    D3D12_DESCRIPTOR_HEAP_DESC desc ;
    desc.Type = D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV;
    desc.NumDescriptors = 1;
    desc.Flags = D3D12_DESCRIPTOR_HEAP_FLAG_SHADER_VISIBLE;

    // g_pd3dDevice->lpVtbl can't find CreateDescriptorHeap
    if (g_pd3dDevice->lpVtbl->CreateDescriptorHeap(g_pd3dDevice,&desc, &IID_ID3D12DescriptorHeap, (void**)&g_pd3dSrvDescHeap) != S_OK)
    {
        return false;
    }

    // tbl->CreateDescriptorHeap is still a valid pointer however.
}

As explaining in the comments, after the line D3D12_CPU_DESCRIPTOR_HANDLE rtvHandle = g_pd3dRtvDescHeap->lpVtbl->GetCPUDescriptorHandleForHeapStart(g_pd3dRtvDescHeap); the g_pd3dDevice->lpVtbl pointer changes and points somewhere invalid and I don't understand why.

I am compiling with the following options:

clang.exe -std=c11 -pedantic-errors -g -D CINTERFACE .\main.c

This is not necessarily the cause of your problem, but in standard C, the `void**` casts are incorrect and an incompatible pointer type conversion. The correct way to get a `void**` from an object pointer is to declare a local `void* tmp = &g_pd3dRtvDescHeap;` then pass `&tmp` to the function. — Lundin, Nov 13 '19 at 08:48
@Lundin *"in standard C, the void * * casts are incorrect and an incompatible pointer type conversion"* - which part of the standard enforces this restriction? `6.3.2.3 Pointers` explicitly allows such conversions. — user7860670, Nov 13 '19 at 10:59
@VTT 6.5.16.1 Simple assignment: `the left operand has atomic, qualified, or unqualified pointer type, and (considering the type the left operand would have after lvalue conversion) both operands are pointers to qualified or unqualified versions of compatible types, and the type pointed to by the left has all the qualifiers of the type pointed to by the right;`. Parameter passing to functions is done according to the rules of simple assignment. — Lundin, Nov 13 '19 at 11:28
@VTT As for 6.3.2.3/7 it doesn't mention what will happen when you de-reference the pointer though an incompatible type. It does mention that such pointer conversions may invoke undefined behavior in case of different alignment though. — Lundin, Nov 13 '19 at 11:31

Chuck Walbourn · Accepted Answer · 2019-11-14T17:00:57.860

1

You are hitting a known bug in the C-bindings... See this thread which explains there's a bug with GetCPUDescriptorHandleForHeapStart.

There were similar issues with the C-bindings with Direct3D9Ex. The basic issue is that almost all users use C++ for DirectX since it naturally maps to COM. The C bindings are mostly automatically generated; they are not well tested or maintained.

edited Nov 14 '19 at 17:00

answered Nov 14 '19 at 09:08

Chuck Walbourn

38,259
2
58
81

cool, I'll just work around this one by holding the pointer before the call to this specific function. Thanks a lot! – mbl Nov 14 '19 at 09:12

Problem calling DX12 functions in a program compiled with clang in c11

1 Answers1