1

I spent a couple of days trying to figure this out but no luck.

I'm trying to render camera feed to OpenGL texture. I managed to get it working without using CVOpenGLESTextureCacheCreateTextureFromImage, simply by creating CGImage out of sampleBuffer. I got some pretty bad performances and I need to use the cache.

As reference I used RosyWriter example. I changed stuff around dozen of times but this is what I got right now which to me looks like just the example but more simplified.

I won't paste AV setup code (if need be I can). Here's the delegate method implementation:

- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    size_t frameWidth = CVPixelBufferGetWidth(pixelBuffer);
    size_t frameHeight = CVPixelBufferGetHeight(pixelBuffer);
    CVOpenGLESTextureRef texture = NULL;
    CVReturn err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
                                                                videoTextureCache,
                                                                pixelBuffer,
                                                                NULL,
                                                                GL_TEXTURE_2D,
                                                                GL_RGBA,
                                                                frameWidth,
                                                                frameHeight,
                                                                GL_BGRA,
                                                                GL_UNSIGNED_BYTE,
                                                                0,
                                                                &texture);


    if (!texture || err) {
        NSLog(@"CVOpenGLESTextureCacheCreateTextureFromImage failed (error: %d)", err);
        return;
    }
}

As you can see I dropped the preview queue (tried using it, didn't work out) and anything that I thought was unnecessary.

Properties passed to the function look ok (pixelBuffer has iosurface, frameWidth/Height are correct).

CVOpenGLESTextureCacheCreateTextureFromImage logs

"Failed to create IOSurface image (texture)"

and returns an error

CVOpenGLESTextureCacheCreateTextureFromImage failed (error: -6683)

As I said I tried alot of things, had different errors and sometimes just had a black texture.

I'm just starting in OpenGL so it might be something obvious, in any case it would be great if someone took a look at the issue. I pasted the code I thought I was relevant, if there's need for anything else I'll update.

UPDATE

After adding this line to the AV output setup, I stopped receiving errors but instead got a blank black texture. I suspected it could have something with color range/format, any ideas?

[output setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
Ilija
  • 151
  • 11
  • That particular error seems to be an intermittent and unexplained one somewhere inside the texture caches: https://github.com/BradLarson/GPUImage/issues/968 , http://stackoverflow.com/questions/12646273/cvopenglestexturecachecreatetexturefromimage-returns-error-6683 , http://stackoverflow.com/questions/10047159/cvopenglestexturecachecreatetexturefromimage-return-6683kcvreturnpixelbufferno . All I can say is that I have the texture caches working well within the GPUImageVideoCamera class here: https://github.com/BradLarson/GPUImage , so you could try to see where you differ from that. – Brad Larson Feb 01 '14 at 18:11
  • I'll definitely have a look. Not sure if I'm onto something but I updated the answer, would appreciate if you took a look. – Ilija Feb 01 '14 at 19:13

1 Answers1

1

Ok so the line mentioned in the update solved the error. After that I had a blank image and didn't know what to do. I used the new XCode 5 OpenGL debugger (Capture OpenGL ES frame in the debug window) and saw that there's an error:

Texture #2 uses an unsupported wrap parameter or a non-power of two texture. Use GL_CLAMP_TO_EDGE instead, or resize the texture.

After that it was pretty clear what to do, for start I changed the wrapping by adding:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

And a camera frame showed up but the texture didn't update. After adding release methods everything started working fine.

CFRelease(texture);
CVOpenGLESTextureCacheFlush(videoTextureCache, 0);

Right now I'm unsure if I should make the texture be a power of two by resizing the pixel buffer (will probably receive a performance boost) or just stick as is.

Anyway hope this helps someone as I've spent way too much time figuring this out.

Ilija
  • 151
  • 11
  • You're going to be perfectly fine with the non-power-of-two texture. You'll lose a lot more in the way of performance and quality by scaling your camera feed to try to be power-of-two, and even older devices can render NPOT camera frames in less than a millisecond. – Brad Larson Feb 03 '14 at 17:21
  • Thanks for posting this, it's a great performance boost. @BradLarson do you know how to create a CVPixelBufferRef from existing data in memory? – jjxtra Jan 25 '15 at 04:01