I can't get vImage (Accelerate Framework) to convert 420Yp8_Cb8_Cr8 (planar) to ARGB8888

Question

I'm trying to convert Planar YpCbCr to RGBA and it's failing with error kvImageRoiLargerThanInputBuffer. I tried two different ways. Here're some code snippets. Note 'thumbnail_buffers + 1' and 'thumbnail_buffers + 2' have width and height half of 'thumbnail_buffers + 0' because I'm dealing with 4:2:0 and have (1/2)*(1/2) as many chroma samples each as luma samples. This silently fails (even though I asked for an explanation (kvImagePrintDiagnosticsToConsole).

error = vImageConvert_YpCbCrToARGB_GenerateConversion( 
        kvImage_YpCbCrToARGBMatrix_ITU_R_709_2,
        &fullrange_8bit_clamped_to_fullrange,
        &convertInfo, 
        kvImage420Yp8_Cb8_Cr8, kvImageARGB8888, 
        kvImagePrintDiagnosticsToConsole);



uint8_t BGRA8888_permuteMap[4] = {3, 2, 1, 0};
uint8_t alpha = 255;

vImage_Buffer dest;
error = vImageConvert_420Yp8_Cb8_Cr8ToARGB8888( 
        thumbnail_buffers + 0, thumbnail_buffers + 1, thumbnail_buffers + 2,
        &dest,
        &convertInfo, BGRA8888_permuteMap, alpha, 
        kvImagePrintDiagnosticsToConsole //I don't think this flag works here
        );

So I tried again with vImageConvert_AnyToAny:

vImage_CGImageFormat cg_BGRA8888_format = {
    .bitsPerComponent = 8,
    .bitsPerPixel = 32,
    .colorSpace = baseColorspace,
    .bitmapInfo = 
        kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder32Little,
    .version = 0,
    .decode = (CGFloat*)0,
    .renderingIntent = kCGRenderingIntentDefault
};


vImageCVImageFormatRef vformat = vImageCVImageFormat_Create(
    kCVPixelFormatType_420YpCbCr8Planar,
    kvImage_ARGBToYpCbCrMatrix_ITU_R_709_2,
    kCVImageBufferChromaLocation_Center,
    baseColorspace,
    0);


vImageConverterRef icref = vImageConverter_CreateForCVToCGImageFormat( 
    vformat,
    &cg_BGRA8888_format,
    (CGFloat[]){0, 0, 0},
    kvImagePrintDiagnosticsToConsole,
    &error );


vImage_Buffer dest;
error = vImageBuffer_Init( &dest, image_height, image_width, 8, kvImagePrintDiagnosticsToConsole);
error = vImageConvert_AnyToAny( icref, thumbnail_buffers, &dest, (void*)0, kvImagePrintDiagnosticsToConsole); //kvImageGetTempBufferSize

I get the same error but this time I get the following message printed to the console.

<Error>: kvImagePrintDiagnosticsToConsole: vImageConvert_AnyToAny: srcs[1].height must be >= dests[0].height

But this doesn't make any sense to me. How can my Cb height be anything other than half my Yp height (which is the same as my dest RGB height) when I've got 4:2:0 data? (Likewise for width?) What on earth am I doing wrong? I'm going to be doing other conversions as well (4:4:4, 4:2:2, etc) so any clarification on these APIs would be greatly appreciated. Further, what is my siting supposed to be for these conversions? Above I use kCVImageBufferChromaLocation_Center. Is that correct?

Some new info: Since posting this I saw a glaring error, but fixing it didn't help. Notice that in the vImageConvert_AnyToAny case above, I initialized the destination buffer with just the image width instead of 4*width to make room for RGBA. That must be the problem, right? Nope.

Notice further that in the vImageConvert_* case, I didn't initialize the destination buffer at all. Fixed that too and it didn't help.

So far I've tried the conversion six different ways choosing one from (vImageConvert_* | vImageConvert_AnyToAny) and choosing one from (kvImage420Yp8_Cb8_Cr8 | kvImage420Yp8_CbCr8 | kvImage444CrYpCb8) feeding the appropriate number of input buffers each time--carefully checking that the buffers take into account the number of samples per pixel per plane. Each time I get:

<Error>: kvImagePrintDiagnosticsToConsole: vImageConvert_AnyToAny: srcs[0].width must be >= dests[0].width

which makes no sense to me. If my luma plane is say 100 wide, my RGBA buffer should be 400 wide. Please any guidance or working code going from YCC to RGBA would be greatly appreciated.

Answer 1

Okay, I figured it out--part user error, part Apple bug. I was thinking of the vImage_Buffer's width and height wrong. For example, the output buffer I specified as 4 * image_width, and 8 bits per pixel, when it should have been simply image_width and 32 bits per pixel--the same amount of memory but sending the wrong message to the APIs. The literal '8' on that line blinded me from remembering what that slot was, I guess. A lesson I must have learned many times--name your magic numbers.

Anyway, now the bug part. Making the input and output buffers correct with regards to width, height, pixel depth fixed all the calls to the low-level vImageConvert_420Yp8_Cb8_Cr8ToARGB8888 and friends. For example in the planar YCC case, your Cb and Cr buffers would naturally have half the width and half the height of the Yp plane. However, in the vImageConvert_AnyToAny cases these buffers caused the calls to fail and bail--saying silly things like I needed my Cb plane to have the same dimensions as my Yp plane even for 4:2:0. This appears to be a bug in some preflighting done by Apple before calling the lower-level code that does the work.

I worked around the vImageConvert_AnyToAny bug by simply making input buffers that were too big and only filling Cb and Cr data in the top-left quadrant. The data were found there during the conversion just fine. I made these too-big buffers with vImageBuffer_Init() where Apple allocated the too-big malloc that goes to waste. I didn't try making the vImage_Buffer's by hand--lying about the size and allocating just the memory I need. This may work, or perhaps Apple will crawl off into the weeds trusting the width and height. If you hand make one, you better tell the truth about the rowBytes however.

I'm going to leave this answer for a bit before marking it correct, hoping someone at Apple sees this and fixes the bug and perhaps gets inspired to improve the documentation for those of us stumbling about.