How to overlay GLKView on top of UIImagePickerController

Question

I'm trying to overlay OpenGL ES content, using a GLKView, on top of a live video feed from the camera using a UIImagePickerController. I've read several tutorials, books, and posts but, still can't find the answer. For your reference, some of the posts I've read are here, here and here.

In viewDidLoad I'm doing the following:

// Create an OpenGL ES 2.0 context and provide it to the view
// NOTE: viewOverlay is of class GLKView and a property of the view controller
viewOverlay.context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
viewOverlay.opaque = NO;
viewOverlay.backgroundColor=[UIColor clearColor];

// Make the new context current
[EAGLContext setCurrentContext:viewOverlay.context];

// Create a base effect that provides standard OpenGL ES 2.0
// Shading Language programs and set constants to be used for 
// all subsequent rendering
self.baseEffect = [[GLKBaseEffect alloc] init];
self.baseEffect.useConstantColor = GL_TRUE;
self.baseEffect.constantColor = GLKVector4Make(
  1.0f, // Red
  0.0f, // Green
  0.0f, // Blue
  1.0f);// Alpha

// Set the background color stored in the current context 
glClearColor(0.0f, 0.0f, 0.0f, 0.0f); // background color

And in viewDidAppear I'm doing this:

    UIImagePickerController *uip;
    uip = [[UIImagePickerController alloc] init];
    uip.sourceType = UIImagePickerControllerSourceTypeCamera;
    uip.showsCameraControls = NO;
    uip.toolbarHidden = YES;
    uip.navigationBarHidden = YES;
    uip.wantsFullScreenLayout = YES;
    uip.cameraViewTransform = CGAffineTransformScale(uip.cameraViewTransform, CAMERA_TRANSFORM, CAMERA_TRANSFORM);
   [viewOverlay addSubview:uip.view];
   [viewOverlay sendSubviewToBack:uip.view];
   [viewOverlay bringSubviewToFront:viewOverlay];

If I step through the program, I can see the OpenGL objects get rendered once. But, when UIImagePickerController is added as a sub-view, it's the only thing on screen. If I comment out the last three lines, the OpenGL objects are rendered but, of course the camera isn't visible.

I'd like the camera's video image to be rendered in back of the OpenGL objects I'm drawing. Creating an augmented reality effect. Any and all help would be greatly appreciated!

Mike

score 1 · Accepted Answer · answered Dec 23 '12 at 12:24

1

What do you want your camera to do - just display a camera view for an AR effect, or actually use/manipulate the image data too? If you need anything other than a simple AR display, use AVFoundation.

I've had loads of trouble with this before and read the same posts you have, so I offer two solutions:

1) Use a Container View

In your storyboard file, create a UIViewController with a UIView for the camera and a Container View on top with a segue connected to a GLKViewController. Link your main UIViewController to your camera code and link your GLKViewController to your OpenGL ES code. Both the UIView and the GLKView inside the container will load and run seperately when the UIViewController is visible.

2) App delegate hack

Setup your UIImagePickerController inside the app delegate, specifically the method:

(BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions

Then, add the subview to your window property. This will always make sure that your camera gets loaded onto your window before any views, and ensures it is at the bottom of the layer stack.

answered Dec 23 '12 at 12:24

Ricardo RendonCepeda

3,271
4
23
30

I'm just trying to get the camera view for a simple AR effect. I'm not trying to manipulate anything in the camera view. Thanks very much for the input! I'll try out your suggestions and let you know how it goes. – MikeyE Jan 09 '13 at 04:20
I tried #2 just because it looked like the quickest. It worked like a charm! I'll implement #1 later when I have time. Thanks so much, you saved me so much time!!! – MikeyE Jan 09 '13 at 04:37
No problem, it's just one of those quick & easy tricks. Once you actually need to use the camera input you should definitely move into AVFoundation. – Ricardo RendonCepeda Jan 09 '13 at 10:19
Good to know. I'll definitely do that. Thanks again!! – MikeyE Jan 10 '13 at 23:55
I'm trying to use your 1st answer. Do you know of a good tutorial or documentation anywhere to implement a container? I've used tab and navigation containers before, but not like this. – MikeyE Jan 17 '13 at 03:41
Not really. You might want to look at this thread as a starting point: http://stackoverflow.com/questions/7755498/container-view-controller-examples – Ricardo RendonCepeda Jan 17 '13 at 08:21
Will do. Thanks for the info. I've been working on an alternate method based on your solution #1. I'll post it when I get it working. – MikeyE Jan 20 '13 at 20:28

How to overlay GLKView on top of UIImagePickerController

1 Answers1