DirectX11 DXGI Capture Screen when in full screen

Question

I'm currently trying to develop a personal project where I will light up LEDs according to what's happening on the screen of my computer.

I've tried several solutions to capture my screen, and DirectX11 with DXGI is the fastest way I found to have a good FPS Rate.

My only issue is the following: When in full screen (for example, watching Netflix trogh Win10 App or playing any game in fullscreen), it seems that nothing is captured. I have two functions (one to set up and another one to capture a frame:

Setup Up Function:

 bool DXGIScreenCapturer::Init() {
    int lresult(-1);

    D3D_FEATURE_LEVEL lFeatureLevel;

    HRESULT hr(E_FAIL);
    hr = D3D11CreateDevice(
        nullptr,
        D3D_DRIVER_TYPE_HARDWARE,
        nullptr,
        0,
        gFeatureLevels,
        gNumFeatureLevels,
        D3D11_SDK_VERSION,
        &_lDevice,
        &lFeatureLevel,
        &_lImmediateContext);

    if (FAILED(hr))
        return false;

    if (!_lDevice)
        return false;

    // Get DXGI device
    ComPtr<IDXGIDevice> lDxgiDevice;
    hr = _lDevice.As(&lDxgiDevice);

    if (FAILED(hr))
        return false;

    // Get DXGI adapter
    ComPtr<IDXGIAdapter> lDxgiAdapter;
    hr = lDxgiDevice->GetParent(__uuidof(IDXGIAdapter), &lDxgiAdapter);

    if (FAILED(hr))
        return false;

    lDxgiDevice.Reset();

    UINT Output = 0;

    // Get output
    ComPtr<IDXGIOutput> lDxgiOutput;
    hr = lDxgiAdapter->EnumOutputs(Output, &lDxgiOutput);

    if (FAILED(hr))
        return false;

    lDxgiAdapter.Reset();

    hr = lDxgiOutput->GetDesc(&_lOutputDesc);

    if (FAILED(hr))
        return false;

    // QI for Output 1
    ComPtr<IDXGIOutput1> lDxgiOutput1;
    hr = lDxgiOutput.As(&lDxgiOutput1);

    if (FAILED(hr))
        return false;

    lDxgiOutput.Reset();

    // Create desktop duplication
    hr = lDxgiOutput1->DuplicateOutput(_lDevice.Get(), &_lDeskDupl);

    if (FAILED(hr))
        return false;

    lDxgiOutput1.Reset();

    // Create GUI drawing texture
    _lDeskDupl->GetDesc(&_lOutputDuplDesc);
    // Create CPU access texture
    _desc.Width = _lOutputDuplDesc.ModeDesc.Width;
    _desc.Height = _lOutputDuplDesc.ModeDesc.Height;
    _desc.Format = _lOutputDuplDesc.ModeDesc.Format;
    std::cout << _desc.Width << "x" << _desc.Height << "\n\n\n";

    _desc.ArraySize = 1;
    _desc.BindFlags = 0;
    _desc.MiscFlags = 0;
    _desc.SampleDesc.Count = 1;
    _desc.SampleDesc.Quality = 0;
    _desc.MipLevels = 1;
    _desc.CPUAccessFlags = D3D11_CPU_ACCESS_FLAG::D3D11_CPU_ACCESS_READ;
    _desc.Usage = D3D11_USAGE::D3D11_USAGE_STAGING;

    while (!CaptureScreen(_result));
    _result = cv::Mat(_desc.Height, _desc.Width, CV_8UC4, _resource.pData);
    return true;
}

And the capture function:

bool DXGIScreenCapturer::CaptureScreen(cv::Mat& output)
{
    HRESULT hr(E_FAIL);
    ComPtr<IDXGIResource> lDesktopResource = nullptr;
    DXGI_OUTDUPL_FRAME_INFO lFrameInfo;
    ID3D11Texture2D* currTexture = NULL;

    hr = _lDeskDupl->AcquireNextFrame(999, &lFrameInfo, &lDesktopResource);

    if (FAILED(hr))
        return false;

    if (lFrameInfo.LastPresentTime.HighPart == 0) // not interested in just mouse updates, which can happen much faster than 60fps if you really shake the mouse
    {
        hr = _lDeskDupl->ReleaseFrame();
        return false;
    }

    //int accum_frames = lFrameInfo.AccumulatedFrames;
    //if (accum_frames > 1) {// && current_frame != 1) {
    //                     // TOO MANY OF THESE is the problem
    //                     // especially after having to wait >17ms in AcquireNextFrame()
    //}

    // QI for ID3D11Texture2D
    hr = lDesktopResource.As(&_lAcquiredDesktopImage);

    // Copy image into a newly created CPU access texture
    hr = _lDevice->CreateTexture2D(&_desc, NULL, &currTexture);
    if (FAILED(hr))
    {
        hr = _lDeskDupl->ReleaseFrame();
        return false;
    }
    if (!currTexture)
    {
        hr = _lDeskDupl->ReleaseFrame();
        return false;
    }

    _lImmediateContext->CopyResource(currTexture, _lAcquiredDesktopImage.Get());
    UINT subresource = D3D11CalcSubresource(0, 0, 0);
    _lImmediateContext->Map(currTexture, subresource, D3D11_MAP_READ, 0, &_resource);
    _lImmediateContext->Unmap(currTexture, 0);
    currTexture->Release();
    hr = _lDeskDupl->ReleaseFrame();

    output = _result;
    return true;
}

I don't know dxgi, but if you want to capture the screen on Windows, you can use a win32 command. Here you can see... https://stackoverflow.com/questions/531684/what-is-the-best-way-to-take-screenshots-of-a-window-with-c-in-windows — Thomas, Apr 01 '18 at 13:12
I also see, that in your capture method you create new textures... It would be better to reuse them to get more fps — Thomas, Apr 01 '18 at 13:15
@Thomas I don't think that command will work for me, as I will need the image in memory to run in at least 30 FPS. For now, tha pproach above, runs at 15ms, I will just need to fix the issue of not capturing any image when an app is running in FullScreen. — Ricardo Alves, Apr 01 '18 at 14:46
Rather than full screen, it looks like intentional exclusion of displayed picture from capture. Hence the blackness. — Roman R., Apr 01 '18 at 14:50
Also I would reuse the texture instead of creating new ones... You can change the width and height to a fix value and play a game to check if it runs more slowly while your program is working. Because creating textures I think is intensive for the GPU... — Thomas, Apr 01 '18 at 22:30
I don't think that video can be captured neither in fullscreen nor in window mode. That being said you might try to come up with a hook solution. — VuVirt, Apr 04 '18 at 13:35
@RicardoAlves hook dx 11 video rendering in the win10 Netflix app for instance — VuVirt, Apr 07 '18 at 12:35

DirectX11 DXGI Capture Screen when in full screen

0 Answers0