Retrieving ID3D11Texture2D data to be sent over network

Question

I am modifying the desktop duplication api sample kindly provided by Microsoft to capture the screen and send updates over the network to my application. I know how to actually send the data; my problem is getting the data from the ID3D11Texture2D object.

ID3D11Texture2D* m_AcquiredDesktopImage;
IDXGIResource* desktopResource = nullptr;
DXGI_OUTDUPL_FRAME_INFO FrameInfo;

// Get new frame
HRESULT hr = m_DeskDupl->AcquireNextFrame(500, &FrameInfo, &desktopResource);

// QI for IDXGIResource
hr = desktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void **>(&m_AcquiredDesktopImage));

At this point, I think the screen updates are in m_AcquiredDesktopImage. I need to transmit this data over the wire (as efficiently as possible).

This answer seems to be on the right track, but I'm new to Windows programming, so I need some additional help.

~~This is the only solution I can imagine utilizing IDXGIObject::GetPrivateData~~

score 2 · Accepted Answer · answered Jul 03 '16 at 03:12

2

Private Data are not what you are looking for at all. They are only here to attach custom values to d3d objects.

Once you have the ID3D11Texture2D object you need to read back the image from, you need to create a second one in the staging memory pool from the ID3D11Device (get the original description, change the pool, and remove the binding).

Then, you need to use the ID3D11DeviceContext to copy the texture to your staging one using CopyResource. Then you can use the context Map and Unmap api to read the image.

answered Jul 03 '16 at 03:12

galop1n

8,573
22
36

Thanks. I think I may be able to do this. What format will the resulting data be in? (Since I will need to update corresponding screen areas on the client) Also, some pseudocode would be tremendously helpful. – cilki Jul 03 '16 at 03:30
1

`ID3D11Texture2D::GetDesc` gets you the format. It will be basically the same format because [`ID3D11DeviceContext::CopyResource`](https://msdn.microsoft.com/en-us/library/windows/desktop/ff476392) has limited conversion capabilities: "Must have compatible DXGI formats, which means the formats must be identical or at least from the same type group". – Roman R. Jul 03 '16 at 03:50
@RomanR. That makes sense. I'm confident in that, but I'm having trouble with creating a second ID2D11Texture2D in the "staging memory pool". What methods do I need to accomplish this? I know how to get the old description, but I'm lost with "change the pool and remove the binding." Is this the correct method: `ID3D11Device::CreateTexture2D` – cilki Jul 03 '16 at 04:21
1

Yes, `CreateTexture2D` with `D3D11_USAGE_STAGING`. Also, if you are going to encode (esp. H.264) then you want to use hardware encoder which takes video memory backed texture - should be more efficient than extracting raw video data first. – Roman R. Jul 03 '16 at 04:44
@RomanR. What is the benefit of encoding with H.264? I thought the frames coming out of `AcquireNextFrame()` were just the dirty regions? – cilki Jul 03 '16 at 16:03
1

You stated that you are going to send over network, and you normally compress video for this because raw video is huge in size. You don't have to use H.264 for this of course - screen codec which takes into consideration dirty/moved regions might be more efficient. H.264 on the other hand might offer efficient hardware assisted encoding which is itself a great benefit. – Roman R. Jul 03 '16 at 16:14
I got a good link which does that.. Look for the method `SaveTextureToBmp` https://github.com/Microsoft/graphics-driver-samples/blob/master/render-only-sample/rostest/util.cpp – Trident Apr 18 '17 at 20:10

score 2 · Answer 2 · edited Apr 18 '17 at 20:22

I got a good link which does that.. Look for the method SaveTextureToBmp

[...]

// map the texture
ComPtr<ID3D11Texture2D> mappedTexture;
D3D11_MAPPED_SUBRESOURCE mapInfo;
mapInfo.RowPitch;
hr = d3dContext->Map(
        Texture,
        0,  // Subresource
        D3D11_MAP_READ,
        0,  // MapFlags
        &mapInfo);

if (FAILED(hr)) {
    // If we failed to map the texture, copy it to a staging resource
    if (hr == E_INVALIDARG) {
        D3D11_TEXTURE2D_DESC desc2;
        desc2.Width = desc.Width;
        desc2.Height = desc.Height;
        desc2.MipLevels = desc.MipLevels;
        desc2.ArraySize = desc.ArraySize;
        desc2.Format = desc.Format;
        desc2.SampleDesc = desc.SampleDesc;
        desc2.Usage = D3D11_USAGE_STAGING;
        desc2.BindFlags = 0;
        desc2.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
        desc2.MiscFlags = 0;

        ComPtr<ID3D11Texture2D> stagingTexture;
        hr = d3dDevice->CreateTexture2D(&desc2, nullptr, &stagingTexture);
        if (FAILED(hr)) {
            throw MyException::Make(hr, L"Failed to create staging texture");
        }

        // copy the texture to a staging resource
        d3dContext->CopyResource(stagingTexture.Get(), Texture);

        // now, map the staging resource
        hr = d3dContext->Map(
                stagingTexture.Get(),
                0,
                D3D11_MAP_READ,
                0,
                &mapInfo);
        if (FAILED(hr)) {
            throw MyException::Make(hr, L"Failed to map staging texture");
        }

        mappedTexture = std::move(stagingTexture);
    } else {
        throw MyException::Make(hr, L"Failed to map texture.");
    }
} else {
    mappedTexture = Texture;
}
auto unmapResource = Finally([&] {
    d3dContext->Unmap(mappedTexture.Get(), 0);
    });

    [...]

    hr = frameEncode->WritePixels(
            desc.Height,
            mapInfo.RowPitch,
            desc.Height * mapInfo.RowPitch,
            reinterpret_cast<BYTE*>(mapInfo.pData));
    if (FAILED(hr)) {
        throw MyException::Make(hr, L"frameEncode->WritePixels(...) failed.");
    }

Retrieving ID3D11Texture2D data to be sent over network

2 Answers2