iOS OpenGL using parameters for glTexImage2D to make a UIImage?

Question

I am working through some existing code for a project i am assigned to.

I have a successful call to glTexImage2D like this:
glTexImage2D(GL_TEXTURE_2D, 0, texture->format, texture->widthTexture, texture->heightTexture, 0, texture->format, texture->type, texture->data);

I would like create an image (preferably a CGImage or UIImage) using the variables passed to glTexImage2D, but don't know if it's possible.

I need to create many sequential images(many of them per second) from an OpenGL view and save them for later use.

Should i be able to create a CGImage or UIImage using the variables i use in glTexImage2D?

If i should be able to, how should i do it?

If not, why can't i and what do you suggest for my task of saving/capturing the contents of my opengl view many times per second?

edit: i have already successfully captured images using some techniques provided by apple with glReadPixels, etc etc. i want something faster so i can get more images per second.

edit: after reviewing and adding the code from Thomson, here is the resulting image:

the image very slightly resembles what the image should look like, except duplicated ~5 times horizontally and with some random black space underneath.

note: the video(each frame) data is coming over an ad-hoc network connection to the iPhone. i believe the camera is shooting over each frame with the YCbCr color space

edit: further reviewing Thomson's code I have copied your new code into my project and got a different image as result:

width: 320 height: 240

i am not sure how to find the number of bytes in texture-> data. it is a void pointer.

edit: format and type

texture.type = GL_UNSIGNED_SHORT_5_6_5

texture.format = GL_RGB

Answer 1

Hey binnyb, here's the solution to creating a UIImage using the data stored in texture->data. v01d is certainly right that you're not going to get the UIImage as it appears in your GL framebuffer, but it'll get you an image from the data before it has passed through the framebuffer.

Turns out your texture data is in 16 bit format, 5 bits for red, 6 bits for green, and 5 bits for blue. I've added code for converting the 16 bit RGB values into 32 bit RGBA values before creating a UIImage. I'm looking forward to hearing how this turns out.

float width    = 512;
float height   = 512;
int   channels = 4;

// create a buffer for our image after converting it from 565 rgb to 8888rgba
u_int8_t* rawData = (u_int8_t*)malloc(width*height*channels);

// unpack the 5,6,5 pixel data into 24 bit RGB
for (int i=0; i<width*height; ++i) 
{
    // append two adjacent bytes in texture->data into a 16 bit int
    u_int16_t pixel16 = (texture->data[i*2] << 8) + texture->data[i*2+1];      
    // mask and shift each pixel into a single 8 bit unsigned, then normalize by 5/6 bit
    // max to 8 bit integer max.  Alpha set to 0.
    rawData[channels*i]   = ((pixel16 & 63488)       >> 11) / 31.0 * 255;
    rawData[channels*i+1] = ((pixel16 & 2016)  << 5  >> 10) / 63.0 * 255;
    rawData[channels*i+2] = ((pixel16 & 31)    << 11 >> 11) / 31.0 * 255;
    rawData[channels*4+3] = 0;
}

// same as before
int                    bitsPerComponent = 8;
int                    bitsPerPixel     = channels*bitsPerComponent;
int                    bytesPerRow      = channels*width;
CGColorSpaceRef        colorSpaceRef    = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo           bitmapInfo       = kCGBitmapByteOrderDefault;
CGColorRenderingIntent renderingIntent  = kCGRenderingIntentDefault;

CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, 
                                                          rawData, 
                                                          channels*width*height,
                                                          NULL);
free( rawData );
CGImageRef        imageRef = CGImageCreate(width,
                                           height,
                                           bitsPerComponent,
                                           bitsPerPixel,
                                           bytesPerRow,
                                           colorSpaceRef,
                                           bitmapInfo,
                                           provider,NULL,NO,renderingIntent);

UIImage *newImage = [UIImage imageWithCGImage:imageRef];

The code for creating a new image comes from Creating UIImage from raw RGBA data thanks to Rohit. I've tested this with our original 320x240 image dimension, having converted a 24 bit RGB image into 5,6,5 format and then up to 32 bit. I haven't tested it on a 512x512 image but I don't expect any problems.

Answer 2

You could make an image from the data you are sending to GL, but I doubt that's really what you want to achieve.

My guess is you want the output of the Frame Buffer. To do that you need glReadPixels(). Bare in mind for a large buffer (say 1024x768) it will take seconds to read the pixels back from GL, you wont get more than 1 per second.

Answer 3

You should be able to use the UIImage initializer imageWithData for this. All you need is to ensure that the data in texture->data is in a structured format that is recognizable to the UIImage constructor.

NSData* imageData = [NSData dataWithBytes:texture->data length:(3*texture->widthTexture*texture->heightTexture)];
UIImage* theImage = [UIImage imageWithData:imageData];

The types that imageWithData: supports are not well documented, but you can create NSData from .png, .jpg, .gif, and I presume .ppm files without any difficulty. If texture->data is in one of those binary formats I suspect you can get this running with a little experimentation.