Video captured by Media Foundation is vertical mirrorred

Question

I'm using Media Foundation IMFSourceReaderCallback implementation for grabbing video frames from the camera, and then OpenCV imshow to present the frames in a loop.
However I get the frames vertically flipped...
Is this a bug? Should I set some attribute to avoid this?
Here is my code:

Initialization:

IMFAttributes* pDeviceAttrs, *pReaderAttrs;
        hr = MFCreateAttributes(&pDeviceAttrs, 1);
        if (FAILED(hr)) goto Exit;
        hr = pDeviceAttrs->SetGUID(MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE, MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID);
        if (FAILED(hr)) goto Exit;
//...
// Correct source provider is activated through ActivateObject  
//
        hr = MFCreateAttributes(&pReaderAttrs, 2);
        if (FAILED(hr)) goto Exit;

        pReaderAttrs->SetUnknown(MF_SOURCE_READER_ASYNC_CALLBACK,(IUnknown*)this);
        pReaderAttrs->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, TRUE); 

        hr = MFCreateSourceReaderFromMediaSource(pMediaSource, pReaderAttrs, &m_pReader);
        if (FAILED(hr)) goto Exit;
// Correct profile is set

OnReadSample implementation:

HRESULT hr = S_OK;
        LONG defaultStride = 0;
        LONG stride = 0;
        BYTE *pBuffer = NULL;

        EnterCriticalSection(&m_critSec);
        if (NULL != pSample)
        {
            IMFMediaBuffer* pMediaBuffer;
            DWORD dataSize = 0;
            // In case of a single buffer, no copy would happen
            hr = pSample->ConvertToContiguousBuffer(&pMediaBuffer);
            if (FAILED(hr)) goto Cleanup;
            pMediaBuffer->GetCurrentLength(&dataSize);

            hr = pMediaBuffer->Lock(&pBuffer, &dataSize, &dataSize);
            if (FAILED(hr)) goto Cleanup;

            // todo: use a backbuffer to avoid sync issues
            if (NULL == m_pLatestFrame) m_pLatestFrame = (BYTE*)malloc(dataSize);
            memcpy(m_pLatestFrame, pBuffer, dataSize);
            ++m_frameNumber;

            pMediaBuffer->Unlock();
            pMediaBuffer->Release();
        }
Cleanup:
        LeaveCriticalSection(&m_critSec);

        // Async ReadFrame for the next buffer:
        hr = m_pReader->ReadSample(
            (DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM,
            0,
            NULL,   // actual
            NULL,   // flags
            NULL,   // timestamp
            NULL    // sample
            );
        return hr;

Conversion to cv::image:

void SourceReaderImpl::GetLatestFrame(BYTE** ppLatestFrame)
    {
        EnterCriticalSection(&m_critSec);
        *ppLatestFrame = m_pLatestFrame;
        LeaveCriticalSection(&m_critSec);
    }

void* CameraWrapperImpl::getLatestFrame()
{
    BYTE* pLatestFrame = NULL;
    m_pMfReader->GetLatestFrame(&pLatestFrame);
    return pLatestFrame;
}

void Player::Present()
{
//...
    color = cv::Mat(colorSize,
                CV_8UC3,
                static_cast<unsigned char*>(m_pColorCameraImpl->getLatestFrame()));
cv::imshow(color);
}

Any idea?

Thanks in advance!

No, that's surely a bug between your ears. Bitmaps are normally stored upside down, last scanline is first in memory. Unless biHeight is negative. You are not properly converting it to the CV array, code you didn't post. — Hans Passant, Nov 09 '14 at 11:16
The code you posted is hardly complete to get details, however I suppose that Hans assumption is correct, you are taking bottom to top RGB image and then you treat it as if it was top to bottom. — Roman R., Nov 13 '14 at 07:54
@RomanR.Is there a way in media foundation to know if the captured video is mirrored or not ? I know that it is the bottom up OR top down that how an image is stored, but is it possible to know that upfront (via some api call perhaps?) — kuldeep, Dec 15 '16 at 09:48

score 4 · Accepted Answer · answered Nov 17 '14 at 19:44

4

A bitmap is stored with the last scan line first, so the image will appear upside down. The easiest solution is to call cv::flip

void Player::Present()
{
    //...
    color = cv::Mat(colorSize,
                CV_8UC3,
                static_cast<unsigned char*>(m_pColorCameraImpl->getLatestFrame()));

    cv::Mat corrected;
    flip(color, corrected, 0);
    imshow(corrected);
}

answered Nov 17 '14 at 19:44

cdmh

3,294
2
26
41

thanks, this is what I currently do. I just wondered if I can change something in the configuration without an extra flip. – rkellerm Nov 19 '14 at 10:26

Video captured by Media Foundation is vertical mirrorred

1 Answers1