7

I've struggled for a week with a problem that I can't rotate a face from non-front like the Left Face in this image to front like the Right Face in the same image.

enter image description here

1). I use a tutorial about delaunay at learnopencv website to get all the triangle of the Left Face (about 138 triangles) like this:

...

triangle num 4: [100, 383] | [120, 374] | [112, 385]

triangle num 5: [120, 374] | [100, 383] | [92, 345]

triangle num 6: [136, 396] | [137, 379] | [151, 394]

triangle num 7: [137, 379] | [136, 396] | [118, 395]

triangle num 8: [92, 345] | [86, 414] | [80, 451]

triangle num 9: [86, 414] | [92, 345] | [100, 383]

triangle num 10: [86, 414] | [100, 383] | [118, 395]

triangle num 11: [116, 336] | [120, 374] | [92, 345]

triangle num 12: [120, 374] | [116, 336] | [133, 344]

...

2). and this library: https://github.com/chili-epfl/attention-tracker (which can estimate head pose) to get a 4x4 matrix transformation of head pose of the Left Face, this matrix seems to be a result of the function Rodrigues in opencv.

 0.659585  -0.749291 -0.0592507  0.0591621 

-0.0327269 -0.107385 0.993679    -0.00523744 

-0.750917  -0.653477 -0.0953513  -0.280204 

 0         0         0           1

So I don't know what is the next move to rotate all the triangle from step 1 with the 4x4 matrix from step 2 to get the result of the Right Face. Pls help!

thor
  • 21,418
  • 31
  • 87
  • 173
Rain Maker
  • 152
  • 1
  • 3
  • 6
  • Maybe some code and images would be helpful to get answers ... – tfv Apr 30 '16 at 04:30
  • The code is really long to read, so I think the result from my code here will be more clear. To be more specific, my list of triangle is declare like this: cv::vector > list_tri_points So each item will be a list of 2D points like the result in step1 and the matrix 4x4 at step 2 I store in a variable of type cv::Mat What I want to achieve is somehow transform all triangles in list_tri_points to new list of triangles by using matrix 4x4. And this new list of triangles will present for a front-face – Rain Maker Apr 30 '16 at 09:04

1 Answers1

3

How to project a face from non-frontal view to frontal view is a complex problem, even you have projected landmark position of the face. I suggest use two open source library that i use before.

The first is provided by MIT. this code isn't only face frontalizer and can use for any object. https://bitbucket.org/gbhuang/congealreal

The second library link is below that provided only for face projection.

https://github.com/eranid/adience_align

I think the second library is the most useful.

erfan zangeneh
  • 334
  • 1
  • 2
  • 10
  • Thanks Zangeneh for replying my question. My purpose is to make a face recognition by comparing the ratios between all possible pairs of landmark points of the input image (non-frontal face image) with ratios of the stored image (frontal face image). But with a non-frontal face, the ratios can't be correct, so I must convert it to a frontal face, and then calculate the ratios. But if as you said, that is a complex problem, is there any other solutions to comparing ratios of 2 faces correctly which count to pose difference but without face projection? – Rain Maker May 01 '16 at 10:47
  • If you want to implement face recognition system that can recognize faces in different pose , you have two ways. The first way is using frontalization module and the second is using face pose estimation that tell our if face is non frontal drop it – erfan zangeneh May 01 '16 at 18:12
  • Hi Zangeneh, could you please explain more specific about using pose estimation in face recognition? I still don't get it. Thanks! – Rain Maker May 02 '16 at 04:36