2

I want to encode video using the "Intel® Quick Sync Video H.264 Encoder MFT". If I create IMFSample from system buffers, it works well. Just like following:

IMFMediaBuffer *pBuffer = NULL;
MFCreateMemoryBuffer(cbSize, &pBuffer);
BYTE *pData = NULL;
pBuffer->Lock(&pData, NULL, NULL);
memcpy(pData, bufferIhaveinYYYYUV format, buffer size);
pBuffer->Unlock();
IMFSample *pSample = NULL;
MFCreateSample(&pSample);
pSample->AddBuffer(pBuffer);

Now I'm investigating whether I can feed it ID3D11Texture2D surfaces as input (DXGI_FORMAT_NV12, 1280x720) in order to improve performance. I tried to pass IMFSample instances created with MFCreateVideoSampleFromSurface or MFCreateDXGISurfaceBuffer to IMFTransform::ProcessInput and made multiple experiments (trying different texture creation flags), but the best result was that all input samples were accepted, but no output samples produced. In case it matters, I never actually tried uploading data to the textures, assuming this would not make a difference from textures filled with garbage pixel data.

Am I doing something wrong?

oguz ismail
  • 1
  • 16
  • 47
  • 69
Tao Mark
  • 31
  • 2
  • 1
    Can you give me some help when you have a problem like you? [here](https://stackoverflow.com/questions/48512660/how-to-create-imfsample-from-d11-texture-for-mft-encode)! – donghui.R Jan 30 '18 at 03:09
  • Tao Mark, Were you able to find a solution for this problem? I'm also stuck with the same. I'm currently trying to feed ID3D11Texture2D surface in NV12 format for performance reasons. Could you please direct me to any online resource? – iamrameshkumar Nov 01 '19 at 14:17

1 Answers1

1

You are basically repeating your earlier question but still without adding any code which does not work.

The fact that you can feed normal (in-memory) samples and have the encoder working suggest that you are doing everything about right. Note that in Direct3D mode you are supposed to not only provide Direct3D 9 surfaces or Direct3D 11 textures, but also comply with respective initialization of the MFT. Specifically, the textures and the MFT's internals have to belong to the same Direct3D device and hence the required steps before the streaming starts. It is not only MFCreateDXGISurfaceBuffer which is needed to be called.

In general, the approach is outlined on MSDN in Supporting Direct3D 11 Video Decoding in Media Foundation article. The same equally well applies to encoding scenario. You are expected to use IMFDXGIDeviceManager pointer and you are expected to use MFT_MESSAGE_SET_D3D_MANAGER message. The MFT operates as MSDN suggests and switches to Direct3D 11 mode accepting texture based samples carrying input frame data.

Community
  • 1
  • 1
Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • After using MFT_MESSAGE_SET_D3D_MANAGER message, It works well. @Roman, thank you. – Tao Mark Apr 18 '17 at 08:03
  • I'm exactly doing the same but getting only green screen in the decoder. https://stackoverflow.com/questions/58503662/getting-green-screen-in-ffplay-streaming-desktop-directx-surface-as-h264-vide – iamrameshkumar Nov 01 '19 at 14:32
  • @Ram: Didn't Rudolfs point you to the right direction there? He also linked my MSDN answer where I mention which attribute holds H.264 parameter set data. – Roman R. Nov 01 '19 at 17:11
  • @RomanR., Rudolfs's answer seems convincing, but I'm still kind of lost in the ocean of confusion. I understand that there could be problems with SPS/PPS, but before moving in that direction I just wanted to ensure that I'm accessing the DirectX surface in a proper way as I'm suspecting the output delivered by the DirectX surface. I don't even have an option to verify whether I'm doing it right due to the limited resources. If there is no way to confirm the correctness of the code, I will have to move on and figure out if the samples are getting proper SPS/PPS embedded. – iamrameshkumar Nov 01 '19 at 18:17
  • Also, you could see in the last screenshot that the ffplay had printed the picture properties in the console. Doesn't that mean the picture properties are embedded with the video samples? That has increased my suspicion of the video source. – iamrameshkumar Nov 01 '19 at 18:21