Assuming a scenario where DX12 is being hooked for overlay rendering it looks like the best function to hook is the IDXGISwapChain::Present the same way it was done for DX11. Having this function hooked the swap chain is available and from that the device can be retrieved to create resources. Given those resources it’s possible to record rendering commands too. The problem arises when we are trying to execute the rendering commands as there is no option to retrieve the associated command queue from the swap chain so there is nothing like this:
CComPtr<ID3D12Device> pD3D12Device;
if (pSwapChain->GetDevice(__uuidof(ID3D12Device), (void**)(&pD3D12Device)) == S_OK)
{
pD3D12Device->GetCommandQueueForSwapChain( swapChain )->ExecuteCommandLists(…);
}
The other option would be creating a new command queue to execute on, like this:
CComPtr<ID3D12Device> pD3D12Device;
if (pSwapChain->GetDevice(__uuidof(ID3D12Device), (void**)(&pD3D12Device)) == S_OK)
{
D3D12_COMMAND_QUEUE_DESC queue_desc = {};
queue_desc.Flags = D3D12_COMMAND_QUEUE_FLAG_NONE;
queue_desc.Type = D3D12_COMMAND_LIST_TYPE_DIRECT;
HRESULT commandQueueRes = _device->CreateCommandQueue( &queue_desc, IID_PPV_ARGS( &_commandQueue ) );
_commandQueue->ExecuteCommandLists( ... );
}
This results in an error and subsequent device removal. See the error message below.
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: A command list, which writes to a swap chain back buffer, may only be executed on the command queue associated with that buffer. [ STATE_SETTING ERROR #907: EXECUTECOMMANDLISTS_WRONGSWAPCHAINBUFFERREFERENCE]
The problem is not resolved even if the ID3D12CommandQueue::ExecuteCommandLists is hooked as well because there is no way to retrieve the associated swap chain from the command queue either.
So my question is what is the recommended way to deal with this problem in a scenario where the swap chain creation happens before the hooking could possibly happen?