5

I want to fit an image of a clown like face into a contour of another face (a person). I am detecting the persons face and getting a elliptical-like contour. I can figure out the center, radius, highest, lowest, left-most and right-most points.

How do I fit the clown face (a square image which I can make elliptical by cutting the face out of the empty background of a png and then detecting the contour) into the persons face?

Or at the least, how do I fit a polygon into another polygon.

I can fit a rectangular image into a rectangular contour with ease, but faces aren't that shape.

Python preferable, but C++ is also manageable, thank you.

Edit: Visual representation as requested:

I have

and I want to make it like this:

but I want the clown face to stretch over the guys face and fit within the blue contour.

Miki
  • 40,887
  • 13
  • 123
  • 202
azazelspeaks
  • 5,727
  • 2
  • 22
  • 39

2 Answers2

1

I think the keyword you are looking for is Active Appearance Models. First, you need to fit a model to first face (such as this one), which lays inside the contour. Then, you should fit the same model to the clown face. After that, since you have fitted same model to both faces, you can stretch it as you need.

I haven't use AAM myself and I'm not an expert about it, so my explanation might not be enough or might not be exactly correct, but I'm sure it will give you some insight.

guneykayim
  • 5,210
  • 2
  • 29
  • 61
  • This seems to be a solution but I can't find an understandable implementation of this to be able to do this, could you provide any more information about AAM? – azazelspeaks Sep 24 '15 at 14:27
  • @azazelspeaks Well, you won't find an understandable implementation of AAM before understanding the theory behind it. Besides, AAM is a very known topic, you shouldn't be asking information to me, you can easily search it and find lots of information about it. Anyway, here is the first ever paper about it https://www.cs.cmu.edu/~efros/courses/LBMV07/Papers/cootes-eccv-98.pdf but I suggest you to find some newer papers about the topic and read them too. Also you may want to search about Active Shape Models. – guneykayim Sep 24 '15 at 17:38
0

A simple and good answer to this question is to find the extreme top, bottom, left, and right points on your contour (head) and then resize your mask to match the aspect ration and place it to cover the 4 points.

Because human heads are elliptical you can use fitEllipse() to give you those 4 points. This will automagically fix any problems with the person tilting their head because regardless of the angle you will know which point is top, bottom, left, and right.

The relevant code for finding the ellipse is:

vector<Point> contour;
// Do whatever you are doing to populate this vector

RotatedRect ellipse = fitEllipse(Mat(contour));

There is also an example as well as documentation for RotatedRect.

// Resize your mask with these sizes for optimum fit
ellipse.size.width
ellipse.size.height

You can rotate your image like this.

UPDATE:

You may also want to find the contour's extreme points to know how much you need to scale your image to ensure that all of the face is covered.

Community
  • 1
  • 1
Rick Smith
  • 9,031
  • 15
  • 81
  • 85
  • I tried this already, but it doesn't work for me because if the mask has a square jaw, the face can be seen inside the blue region but outside the mask in the little space in between. – azazelspeaks Sep 24 '15 at 14:25
  • I think what this boils down to is that you will either have to make the mask bigger (or smaller) than the face or you will have to distort the shape of the mask (perhaps different amounts in different places) to fit the face. There are tradeoffs to both ways and both will sometimes look a little funny. – Rick Smith Sep 24 '15 at 16:34