2

I'm trying to convert RGB32 frames to NV12 Frames to feed into an encoder.

m_iWidthIn = 1920;
m_iHeightIn = 1080;
m_iWidthOut = (((iWidthIn  + 31) >> 5) << 5) //32bit align
m_heightOut = (((iHeightIn + 31) >> 5) << 5) //32bit align
m_outputPixelFormat = AV_PIX_FMT_NV12;

// allocate and fill buffers

m_sws = ::sws_getContext(m_iWidthIn, m_iHeightIn, AV_PIX_FMT_RGB32, m_iWidthOut, m_iHeightOut, m_outputPixelFormat, SWS_FAST_BILINEAR, nullptr, nullptr, nullptr);
AVFrame* frameOut = av_frame_alloc();
frameOut->height = m_iHeightOut;
frameOut->width = m_iWidthOut;
frameOut->format = m_outputPixelFormat;
av_frame_get_buffer(frameOut, 32);
int linesize[1] = { m_iWidthIn * 4 };
uint8_t * data[1] = { m_inputBuffer  };
if (m_bFlip)
{
    data[0] += linesize[0] * (m_iHeightIn - 1);
    linesize[0] = -linesize[0];
}
::sws_scale(m_sws, data, linesize, 0, m_iHeightIn, frameOut->data, frameOut->linesize);
::av_image_copy_to_buffer(pOutputBuffer, lDataLen, frameOut->data, frameOut->linesize, m_outputPixelFormat, m_iWidthOut, m_iHeightOut, 32);

If I make m_outputPixelFormat AV_PIX_FMT_RGB32 and use a DMO colorspace converter, the video comes out correctly. However if I change it to NV12, I end up with a slanted video with missing data at the bottom. I know this is caused by me copying the data incorrectly out of the buffer, but I'm unsure what I'm doing incorrectly.

KevinA
  • 619
  • 6
  • 25
  • Your `frameOut->data` should have two pointers for NV12, for Y and for UV data. Similarly, `linesize`. I don't see you are getting these preparations in the code snippet above. – Roman R. Mar 16 '16 at 15:29
  • `av_image_copy_to_buffer` is an avutil function. I would think it should work with a supported format. – KevinA Mar 16 '16 at 18:30
  • You need to provide planes in the `sws_scale` call, when the one you mentioned, `av_image_copy_to_buffer` is only going to take place afterwards. – Roman R. Mar 17 '16 at 11:44

1 Answers1

1

Your problem is here:

m_heightOut = (((iHeightIn + 31) >> 5) << 5) //32bit align

You don't need to align height. So frameOut->data has m_iHeightIn height. The correct line is:

m_heightOut = iHeightIn;
user1554270
  • 46
  • 9