Convert UIImage to CVImageBufferRef

Question

This code mostly works, but the resulting data seems to loose a color channel (is what I am thinking) as the resulting image data when displayed is tinted blue!

Here is the code:

UIImage* myImage=[UIImage imageNamed:@"sample1.png"];
CGImageRef imageRef=[myImage CGImage];
CVImageBufferRef pixelBuffer = [self pixelBufferFromCGImage:imageRef];

The method pixelBufferFromCGIImage was grabbed from another post on stackoverflow here: How do I export UIImage array as a movie? (although this application is unrelated to what I am trying to do) it is

+ (CVPixelBufferRef)pixelBufferFromCGImage:(CGImageRef)image
{
    CGSize frameSize = CGSizeMake(CGImageGetWidth(image), CGImageGetHeight(image));
    NSDictionary *options = @{
                              (__bridge NSString *)kCVPixelBufferCGImageCompatibilityKey: @(NO),
                              (__bridge NSString *)kCVPixelBufferCGBitmapContextCompatibilityKey: @(NO)
                              };
    CVPixelBufferRef pixelBuffer;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, frameSize.width,
                                          frameSize.height,  kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
                                          &pixelBuffer);
    if (status != kCVReturnSuccess) {
        return NULL;
    }

    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    void *data = CVPixelBufferGetBaseAddress(pixelBuffer);
    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(data, frameSize.width, frameSize.height,
                                                 8, CVPixelBufferGetBytesPerRow(pixelBuffer), rgbColorSpace,
                                                 (CGBitmapInfo) kCGImageAlphaNoneSkipLast);
    CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
                                           CGImageGetHeight(image)), image);
    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);
    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

    return pixelBuffer;
}

I am thinking it has something to do with the relationship between kCVPixelFormatType_32ARGB and kCGImageAlphaNoneSkipLast though I have tried every combination and get either the same result or a application crash. Once again, this gets the UIImage data into CVImageBufferRef but when I display the image on screen, it appears to loose a color channel and shows up tinted blue. The image is a png.

score 5 · Answer 1 · answered Nov 16 '11 at 00:39

5

The solution is that this code works perfectly as intended. :) The issue was in using the data in creating an OpenGL texture. Completely unrelated to this code. Anyone searching for how to Convert UIImage to CVImageBufferRef, your answer is in the above code!

answered Nov 16 '11 at 00:39

Eric

1,014
2
13
22

No, it most definitely doesn't work. You need to use `kCVPixelFormatType_32BGRA`, and use `(CGBitmapInfo) kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst` as the last variable for the `context`. What's more, the `NSDictionary *options` are useless IMO – Maxi Mus Nov 30 '16 at 17:34

score 1 · Answer 2 · answered Jul 20 '15 at 14:01

If anyone is still looking for a solution to this problem, I solved it by switching the BOOLs in the pixelBuffer's options:

NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                     [NSNumber numberWithBool:NO], kCVPixelBufferCGImageCompatibilityKey,
                     [NSNumber numberWithBool:NO], kCVPixelBufferCGBitmapContextCompatibilityKey,
                     nil];

From NO to YES:

NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                     [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                     [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                     nil];

score 1 · Answer 3 · answered Mar 18 '16 at 07:32

1

I encounter the same problem and find some samples: http://www.cakesolutions.net/teamblogs/2014/03/08/cmsamplebufferref-from-cgimageref
try to change

CGBitmapInfo  bitmapInfo = (CGBitmapInfo)kCGBitmapByteOrder32Little |
                  kCGImageAlphaPremultipliedFirst)

answered Mar 18 '16 at 07:32

JoShin

71
3
10

For me this worked! just so people can see the code part of it, CGContextRef context = change the value passed in from that from kCGImageAlphaNoneSkipLast to kCGImageAlphaPremultipliedFirst. – Charlie Apr 22 '16 at 19:51

score 1 · Answer 4 · answered Nov 30 '16 at 18:26

Here's what really works:

+ (CVPixelBufferRef)pixelBufferFromImage:(CGImageRef)image {
    CGSize frameSize = CGSizeMake(CGImageGetWidth(image), CGImageGetHeight(image)); // Not sure why this is even necessary, using CGImageGetWidth/Height in status/context seems to work fine too

    CVPixelBufferRef pixelBuffer = NULL;
    CVReturn status = CVPixelBufferCreate(kCFAllocatorDefault, frameSize.width, frameSize.height, kCVPixelFormatType_32BGRA, nil, &pixelBuffer);
    if (status != kCVReturnSuccess) {
        return NULL;
    }

    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    void *data = CVPixelBufferGetBaseAddress(pixelBuffer);
    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(data, frameSize.width, frameSize.height, 8, CVPixelBufferGetBytesPerRow(pixelBuffer), rgbColorSpace, (CGBitmapInfo) kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
    CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image), CGImageGetHeight(image)), image);

    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);
    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

    return pixelBuffer;
}

You can change the pixel buffer back to a UIImage (and then display or save it) to confirm that it works with this method:

+ (UIImage *)imageFromPixelBuffer:(CVPixelBufferRef)pixelBuffer {
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];
    CIContext *context = [CIContext contextWithOptions:nil];
    CGImageRef myImage = [context createCGImage:ciImage fromRect:CGRectMake(0, 0, CVPixelBufferGetWidth(pixelBuffer), CVPixelBufferGetHeight(pixelBuffer))];
    UIImage *image = [UIImage imageWithCGImage:myImage];

    // Uncomment the following lines to say the image to your application's document directory
    //NSString *imageSavePath = [documentsDirectory stringByAppendingPathComponent:[NSString stringWithFormat:@"myImageFromPixelBuffer.png"]];
    //[UIImagePNGRepresentation(image) writeToFile:imageSavePath atomically:YES];
    return image;
}

score 0 · Answer 5 · answered Jun 15 '12 at 01:05

Just to clarify the answer above: I've ran into the same issue because my shader code was expecting two layered samples within a image buffer, while I used a single layer buffer

This line took the rgb values from one sample and passed them to (I don't know what), but the end result is full colored image.

 gl_FragColor = vec4(texture2D(SamplerY, texCoordVarying).rgb, 1);

score -1 · Accepted Answer · answered Nov 15 '11 at 14:40

-1

It sounds like it might be that relationship. Possibly have it be a jpg and RGB instead of indexed colors with a png?

answered Nov 15 '11 at 14:40

Nathan M Rose

14
2

2

Thanks for the answer. The solution is that this code works perfectly as intended. The issue was in using the data in creating an OpenGL texture. Completely unrelated to this code. Anyone searching for how to Convert UIImage to CVImageBufferRef, your answer is in the above code! – Eric Nov 15 '11 at 15:54
For reference: PNG is not necessarily indexed (like GIF), it also supports lossless full-color RGB images (obviously this works better with line drawings and solid color blocks) – Ethan Aug 26 '14 at 18:50

Convert UIImage to CVImageBufferRef

6 Answers6

Linked