How to convert a kCVPixelFormatType_420YpCbCr8BiPlanarFullRange Buffer to YUV420 using libyuv library in ios?

Question

i have captured video using AVFoundation .i have set (video setting )and get in outputsamplebuffer kCVPixelFormatType_420YpCbCr8BiPlanarFullRange format. But i need YUV420 format for further processing.

For that i use libyuv framework.

LIBYUV_API
int NV12ToI420(const uint8* src_y, int src_stride_y,
           const uint8* src_uv, int src_stride_uv,
           uint8* dst_y, int dst_stride_y,
           uint8* dst_u, int dst_stride_u,
           uint8* dst_v, int dst_stride_v,
           int width, int height);

 libyuv::NV12ToI420(src_yplane, inWidth ,
                   src_uvplane, inWidth,
                   dst_yplane, inWidth,
                   dst_vplane, inWidth / 2,
                   dst_uplane, inWidth / 2,
                   inWidth,  inHeight);

But i am getting output buffer is full green color? i done any mistake for that process pls help me?

You mean `kCVPixelFormatType_420YpCbCr8BiPlanarFullRange` is a format name? Dear God... — Tomasz Kowalczyk, Feb 26 '13 at 03:43

score 0 · Answer 1 · answered Feb 26 '13 at 03:42

0

Looks right. Make sure your src_uvplane points to src_yplane + inWidth * inHeight

answered Feb 26 '13 at 03:42

phthoruth

26
1

Tsung Wu · Answer 2 · 2013-04-07T06:23:40.530

You need convert your data to I420, I am processing camera too, but on Android. I think it should be similar on iOS. Android raw camera is NV21 or NV16 format, I convert from NV21 or NV16 to YV12, I420 is almost the same as YV12:

BYTE m_y[BIG_VIDEO_CX * BIG_VIDEO_CY], 
     m_u[(BIG_VIDEO_CX/2) * (BIG_VIDEO_CY/2)],
     m_v[(BIG_VIDEO_CX/2) * (BIG_VIDEO_CY/2)];

    void NV21_TO_YV12(BYTE *data)
{
    int width = BIG_VIDEO_CX;
    int height = BIG_VIDEO_CY;
        m_y2=data;
    data=&data[width*height];
    for (uint32_t i=0; i<(width/2)*(height/2); ++i)
    {
        m_v[i]=*data;
        m_u[i]=*(data+1);
        data+=2;
    }
}
void NV16_TO_YV12(BYTE *data)
{
    int width = BIG_VIDEO_CX;
    int height = BIG_VIDEO_CY;
    m_y2=data;
    const BYTE* src_uv = (const BYTE*)&data[width*height];
    BYTE* dst_u = m_u;
    BYTE* dst_v = m_v;
    for (uint32_t y=0; y<height/2; ++y)
    {
        const BYTE* src_uv2 = src_uv + width;
        for (uint32_t x=0; x<width/2; ++x)
        {
            dst_u[x]=(src_uv[0]+src_uv2[0]+1)>>1;
            dst_v[x]=(src_uv[1]+src_uv2[1]+1)>>1;
            src_uv+=2;
            src_uv2+=2;
        }
        src_uv=src_uv2;
        dst_u+=width/2;
        dst_v+=width/2;
    }
}

score 0 · Answer 3 · answered May 30 '13 at 06:14

Android is NV21, which libyuv supports with Arm as well as Intel. It can also rotate by 90, 180 or 270 as part of the conversion if necessary for orientation. The Arm optimized version is about 2x faster than C

C
NV12ToI420_Opt (782 ms)
NV21ToI420_Opt (764 ms)

Arm (Neon optimized)
NV12ToI420_Opt (398 ms)
NV21ToI420_Opt (381 ms)

Curious you use NV16 on Android. I'd expect NV61 for consistency with NV21. Your code looks correct, but would nicely optimize into Neon using vrhadd.u8. File a libyuv issue if you'd like to see that. https://code.google.com/p/libyuv/issues/list

Aki · Answer 4 · 2013-09-24T16:10:24.963

Here is how I do it on iOS in my captureOutput after I get a raw video frame from AVCaptureSession(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange):

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection 
{

    CVImageBufferRef videoFrame = CMSampleBufferGetImageBuffer(sampleBuffer);

    CFRetain(sampleBuffer);

    CVPixelBufferLockBaseAddress(videoFrame, 0);
    size_t _width = CVPixelBufferGetWidth(videoFrame);
    size_t _height = CVPixelBufferGetHeight(videoFrame);

    const uint8* plane1 = (uint8*)CVPixelBufferGetBaseAddressOfPlane(videoFrame,0);
    const uint8* plane2 = (uint8*)CVPixelBufferGetBaseAddressOfPlane(videoFrame,1);
    size_t plane1_stride = CVPixelBufferGetBytesPerRowOfPlane (videoFrame, 0);
    size_t plane2_stride = CVPixelBufferGetBytesPerRowOfPlane (videoFrame, 1);

    size_t plane1_size = plane1_stride * CVPixelBufferGetHeightOfPlane(videoFrame, 0);
    size_t plane2_size = CVPixelBufferGetBytesPerRowOfPlane (videoFrame, 1) * CVPixelBufferGetHeightOfPlane(videoFrame, 1);

    size_t frame_size = plane1_size + plane2_size;

    uint8* buffer = new uint8[ frame_size ];
    uint8* dst_u = buffer + plane1_size;
    uint8* dst_v = dst_u + plane1_size/4;

    // Let libyuv convert
    libyuv::NV12ToI420(/*const uint8* src_y=*/plane1, /*int src_stride_y=*/plane1_stride,
                /*const uint8* src_uv=*/plane2, /*int src_stride_uv=*/plane2_stride,
                   /*uint8* dst_y=*/buffer, /*int dst_stride_y=*/plane1_stride,
                   /*uint8* dst_u=*/dst_u, /*int dst_stride_u=*/plane2_stride/2,
                   /*uint8* dst_v=*/dst_v, /*int dst_stride_v=*/plane2_stride/2,
                   _width, _height);

    CVPixelBufferUnlockBaseAddress(videoFrame, 0);
    CFRelease( sampleBuffer)

    // TODO: call your method here with 'buffer' variable. note that you need to deallocated the buffer after using it
  }

I made the code a bit more descriptive for clarity.

No need to retain and release sampleBuffer just lock and unlock after use — Voda Ion, Sep 17 '13 at 09:08
i tweaked the code, so it is clear that the buffer needs to be deallocated after it is used — Aki, Sep 24 '13 at 16:11

How to convert a kCVPixelFormatType_420YpCbCr8BiPlanarFullRange Buffer to YUV420 using libyuv library in ios?

4 Answers4

Linked