21

I'm trying to get a similar effect to the one described in the pictures. The white area on the left isn't on the original photo, it pushes the pixels away. I've tried using Core Image (which offered a few close effects to this) but it was too slow for my needs. I need the effect to be snappy so I can make it responsive to touch.

Is there a close effect \ animation in the iOS sdk that does something similar to this with a UIVIew? ( I couldn't find anything close) If not, can someone offer how to attack this using OpenGL? (code examples are more than welcome)

(This effect will take an important roll in an open source class I'm writing so I'd prefer not to use a 3rd party class such as GPUImage if a simpler solution is available (I'm targeting this class to be dependencies free))

enter image description hereenter image description hereenter image description here

Segev
  • 19,035
  • 12
  • 80
  • 152
  • Will it be OK if the white are is just cut from the image? Or you need the rest of the image to stretch along with it? – Kamran Khan Jan 06 '14 at 10:47
  • @kamran The white area is just the background. After the stretching effect I need the complete image stretched without cutting anything out of it. – Segev Jan 06 '14 at 10:52
  • Have you tried using CATransform3D with UIView animation or CABasicAnimation on the image? – Kamran Khan Jan 06 '14 at 11:28
  • @kamran Yes. I couldn't get the desired behavior. – Segev Jan 06 '14 at 11:39
  • yes, I am also testing with your image with CATransform3D but its not working i guess. Let me test a bit more, if it doesn't work then only thing that you can do is to use layer and layer masks to achieve your desired purpose, but in that case the image might get cropped. – Kamran Khan Jan 06 '14 at 11:42
  • http://wdnuon.blogspot.com/2010/05/implementing-ibooks-page-curling-using.html check this, if same technique can be used. – Kamran Khan Jan 06 '14 at 11:48
  • Also check this http://www.techotopia.com/index.php/An_iOS_4_iPhone_Graphics_Drawing_Tutorial_using_Quartz_2D#Drawing_an_Ellipse_or_Circle – Kamran Khan Jan 06 '14 at 11:52
  • @karmran, the article on page curls is the sort of thing you need to do: mapping a texture onto a mesh and then distorting the mesh. This requires OpenGL. There's no way to do it with a simple 3D transform. The best you could get with a transform is a skew. – Duncan C Jan 10 '14 at 04:03
  • Did u try the ImageMagick C APIs. Maybe this can help you out - http://stackoverflow.com/questions/11348415/image-bending-transform-in-ios – Sj. Jan 10 '14 at 19:48

5 Answers5

27

Sha,

What I did was to lay out a rectangular grid of points in my OpenGL coordinates. (I used a 50x50 grid). I then define one or more control points in that same coordinate space. I have a general pinch-stretch algorithm that I wrote. It takes a control point, a radius, and a signed integer for the amount to attract/repel grid points, and applies it to the coordinates of my grid. For a pinch, it pulls the grid points closer to the control point, and for a stretch, it pushes the control points away. The change is greatest at the control point and goes down as the distance from the control point increases up to the radius value.

Once I've changed the coordinates of my mesh, I then define a set of triangle strips that draws the entire (warped) rectangular area of my image. I can render my entire texture onto the warped mesh with one OpenGL drawing call. OpenGL takes care of stretching the image pixels to fit the distorted triangles in my mesh of control points. Here is a sample before/after image that uses several control points to create a fat-face look. It shows the mesh grid so you can see what it's doing. It's my ugly mug, so apologies in advance:

enter image description here

And the stretched image:

enter image description here

In my case I wanted the control points along the perimeter of the image to stay in place and only the interior of the image to be distorted, so I added extra code logic to lock the perimeter control points into place. That's why the frame of the image is not pushed in like in your image. That required extra code however.

The code ends up using the distance formula to figure out how far each grid-point is from a control point, and then trig to figure out the angle, multiplication to change the distance, and then more trig to calculate the new location for each grid-point. Since I'm only transforming a 50x50 grid of control points, though, its fast enough to keep up.

Our app, Face Dancer, is free on the app store. (Download it at this link: Face Dancer in the App store You might want to download it and try it out. (It includes a number of free morphs, and then offers additional morphs as in-app purchases) There is also a paid "Plus" version that has all the morphs included.

When you're running Face Dancer if you two finger double-tap on the image, it displays the grid of control points it's using to create the morph so you can see what it's doing.

There is an OpenGL sample app called GLCameraRipple that is included with Xcode and linked from the documentation. I would suggest taking a look at that as a starting point. It takes a video feed from the camera and draws it to the screen. If you touch the screen it distorts the image as if the screen were a pool of water and your finger was causing ripples in it. It uses a mesh warp technique like what I described.

Unfortunately I don't think the app uses GLKit, which is a much easier way to use OpenGL ES 2.0. GLKit offers a number of features that make it much easier to use, including GLKViews and GLKViewControllers.

Koedlt
  • 4,286
  • 8
  • 15
  • 33
Duncan C
  • 128,072
  • 22
  • 173
  • 272
  • Thanks for a great detailed answer Duncan. While researching on this matter (before posting this question) I came across one of your explanations around the web and already downloaded Face Dancer then to have a better understanding on the matter. I hoped there's a way to avoid OpenGL with something simple like a core animation (there's a very nice private API "ripple" animation so I hoped for something similar for warping) but I guess there's no other way...OpenGL here I come. – Segev Jan 07 '14 at 19:47
  • My pleasure. Does that meet your criteria for the bounty? – Duncan C Jan 07 '14 at 22:59
  • I'll wait for the bounty to run out just incase someone will come up with a creative idea on how to avoid OpenGL. If not, your answer definitely meets the criteria for the bounty. – Segev Jan 08 '14 at 06:15
  • Duncan, check out the open source class posted in the answer I just added. I haven't dived into the code yet but from skimming over the project it looks like a great example of what you were describing above. – Segev Jun 15 '14 at 12:08
  • @Segev, that looks really cool. I've downloaded the project and hope to spend some time looking at it soon. – Duncan C Jun 15 '14 at 22:27
5

There's a new great open source class that answers my question completely with a magnificent example. You can find it here.

enter image description here

Segev
  • 19,035
  • 12
  • 80
  • 152
1

Another thing you might look at is the Core Image hole distortion filter (CIHoleDistortion) that was added in iOS 6. It creates an effect rather like what you're looking for. Here is a sample image generated from a test program I wrote that gets pretty close.

However, you'll notice that Apple's filter causes the image to "pull away" from the edges of the frame. I think this is a bug, but good luck getting Apple to fix it:

enter image description here

Duncan C
  • 128,072
  • 22
  • 173
  • 272
  • Thanks Duncan. When I've researched on this matter I've found `CIHoleDistortion` and while testing it it didn't met my demands. I'm not sure what it was( maybe the symmetricity of the circle while I was going for something more abstract?) Any way i'm already hard & heavy into OpenGL ES :) Thanks again for updating this question with great answers! – Segev Mar 25 '14 at 17:07
0

What you show looks like a mesh warp. That would be straightforward using OpenGL, but "straightforward OpenGL" is like straightforward rocket science.

I wrote an iOS app for my company called Face Dancerthat's able to do 60 fps mesh warp animations of video from the built-in camera using OpenGL, but it was a lot of work. (It does funhouse mirror type changes to faces - think "fat booth" live, plus lots of other effects.)

Duncan C
  • 128,072
  • 22
  • 173
  • 272
  • Could you include some more detail on the challenges you faced, and how you generally overcame them? – Aaron Brager Dec 31 '13 at 23:25
  • The only effect I'm looking for is the above. If OpenGL is my only option I'll start learning how to accomplish that. Is there any good tutorial anyone knows that will focus on the above without all the clutter around it? – Segev Jan 01 '14 at 06:43
  • I don't know of any other way to do this sort of transformation other than OpenGL or some other approach that is built on top of OpenGL. What you want to do is to map your image into an OpenGL texture, then cut the texture into a series of tessellated triangles in a "mesh grid". When you want to warp your image, you push the points of the mesh in, and draw your texture onto the now-distorted mesh of triangles. – Duncan C Jan 06 '14 at 22:58
  • @Aaron, for me the biggest challenges were: a) Figuring out how to move pixels through the system, into OpenGL, and then out to the screen fast enough to keep up with the video framerate. I found a sample app from 2011 WWDC called Chromakey that does some tricks with CVPixelBuffers so the video frames stay in GPU memory with no buffer copying. b) Dealing with OpenGL setup and configuration. OpenGL is a big, complex API with lots of fussy setup. If you don't do it all the time it's pretty overwhelming, and until you get it right your code fails with unhelpful OpenGL errors. – Duncan C Jan 07 '14 at 14:43
0

I would investigate two approaches: First, Core Image Filters, which include a number of distortion filters like you seem to need. The main problem is that many of these aren't available in iOS (but are on OSX...).

Second, the GPUImage library has a number of distortion filters that you can use too. Here's a Q that discusses how they might be used:

How can you apply distortions to a UIImage using OpenGL ES?

Community
  • 1
  • 1
TomSwift
  • 39,369
  • 12
  • 121
  • 149
  • GPUImage would be a good way to go, but the OP says he doesn't want to use third party frameworks. To the OP, understand that code licensed under a BSD or similar source license lets you use the code for commercial purposes, and even re-sell frameworks that include it to other clients. You should expect to spend a couple of weeks (at least!) coming up to speed on OpenGL and getting this accomplished if you do it yourself. OpenGL has a HUGE learning curve. – Duncan C Jan 10 '14 at 17:32