3

I'm having difficulties finding any documentation about cropping images using OpenGL ES on the iPhone or iPad.

Specifically, I am capturing video frames at a mildly rapid pace (20 FPS), and need something quick that will crop an image. Is it feasible to use OpenGL here? If so, will it perform faster than cropping using Core Image and its associated methods?

It seems that using Core Image methods, I can't achieve faster than about 10-12 FPS output, and I'm looking for a way to hit 20. Any suggestions or pointers to usage of OpenGL for this?

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
anon_dev1234
  • 2,143
  • 1
  • 17
  • 33
  • Rather than do all the OpenGL ES code yourself, you might want to take a look at this: https://github.com/BradLarson/GPUImage . In particular, the GPUImageCropFilter should do what you want. – Brad Larson Mar 01 '13 at 03:26
  • @BradLarson Hey yea I took a good look at your code and I have to say I am very impressed by your work. The only thing I was conflicted about in regards to using your implementation was whether or not it would work with a real-time video feed - simply because it is classified as an image filter. I wasn't sure if it was intended to be used in a way I need it to be. Would you say it could be used in such a manner? Would definitely save me a lot of headache.. – anon_dev1234 Mar 01 '13 at 04:33
  • 1
    It was written with the sole purpose of operating on live video from the camera. I have some benchmarks here: http://stackoverflow.com/a/6628208/19679 (which I need to update for iOS 6.0). Build and run the FilterShowcase example to see how this performs for live video. The crop filter is probably the fastest of the bunch, so it should crop and record 720p video at ~25-30 FPS on an iPhone 4, and 1080p video at a solid 30 FPS on 4S and above. – Brad Larson Mar 01 '13 at 04:38
  • Wow. That is just awesome. I don't even need 720p video, I'm using the 352x288 preset for image size reduction. You are a lifesaver! – anon_dev1234 Mar 01 '13 at 04:44
  • @BradLarson Actually after a lot of fidgeting and testing, I don't know that GPUImage helps here. The face detection just slows the device down too much. I'm not cropping a static region - it's dynamic and determined by the `CIFaceDetector` results. If you notice on your face detection in the samples, the box has a slow framerate when rendering. I'm chalking this up to just the slowness of `CIFaceDetector`, and not any image manipulation I was doing. Which is tragic, but oh well :( – anon_dev1234 Mar 01 '13 at 21:54
  • Yes, the Core Image face detection logic is fairly slow, but you can still crop in realtime based on the last known location of the faces in the scene. Unfortunately, I don't yet have my own facial detection logic working yet. – Brad Larson Mar 01 '13 at 21:56

1 Answers1

3

Obviously, using OpenGl ES will be faster than Core Image Framework. Cropping image will be done by set Texture Coordinate, in generally, Texture Coordinate always like this,

{
0.0f,1.0f,
1.0f,1.0f,
0.0f,0.0f,
1.0f.0.0f
}

The whole image will be drawed with Texture Coordinate above. If you just want upper right part of a image, you can set Texture Coordinate like this,

{
0.5f,1.0f,
1.0f,1.0f,
0.5f,0.5f,
1.0f.0.5f
}

This will get a quater of the whole image at upper right. You never forget that the Coordinate origin of OpenGl ES is at the lower left corner

zhzhy
  • 461
  • 3
  • 17
  • Ok, now assuming I have a rect computed from the `CIFaceDetector`, I guess I would just normalize the coordinates? I've never really used OpenGL so are there any resources you could point me to where I can look to actually render the new texture from the old texture and a set of coordinates? – anon_dev1234 Feb 28 '13 at 01:58
  • 1
    @bgoers Yes, you first normalize the rect got from CIFaceDetector against to the origin size of texture, then you can get the Texture Coordinate. How to render texture with OpenGl ES you can get help from my open source program **Camera** at https://github.com/zhzhy/Camera, or download demo from developer.apple.com, such as RosyWriter, GLCameraRipple and so on. These program will help you learn to be familiar with the usage of OpenGl ES. – zhzhy Feb 28 '13 at 02:18
  • Thank you. I'll have a look at it. I'm going to have to read up on OpenGL for a bit to understand some of the functions! – anon_dev1234 Feb 28 '13 at 02:47