7

Can I get some ideas on how to morph the face in a live video using opencv? I have tried Face substitution but it is implemented using openFrameworks.

I would like to implement the same using opencv. Is there any other methods available in opencv than diirectly porting Face substituion code from openFrameworks to Opencv?

I have also gone through this link, but few people have mentioned as the face morphing is deprecated in opencv?

Sandipan Dey
  • 21,482
  • 2
  • 51
  • 63
2vision2
  • 4,933
  • 16
  • 83
  • 164
  • image morphing (your blog link) is completely different from live video morphing - and opencv will never do everything what openframeworks does, that's why your github project uses BOTH – Gabor Nov 17 '14 at 19:57
  • Maybe you can use calcOpticalFlowFarneback. See this youtube link: https://www.youtube.com/watch?v=F4k9q-HXT7Y Also, see the opencv dense optical flow tutorial (at bottom of page): http://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_lucas_kanade/py_lucas_kanade.html. – user391339 Feb 12 '15 at 08:43

4 Answers4

11

I recently wrote an article on face morphing using OpenCV. I have shared the code in C++ / Python. You can find the details here

http://www.learnopencv.com/face-morph-using-opencv-cpp-python/

but the basic idea is as follows.

  1. Find Point Correspondences in the two input images.
  2. Do Delaunay triangulation on the points.
  3. The amount of morphing is controlled by a parameter alpha. E.g .for alpha = 0, you will get Ted Cruz in the example below, and for alpha = 1. you will get Hillary Clinton. For any other alpha, it will be the blend between the two. Use alpha to calculate the location of the points in the output ( morphed ) image by taking a weighted average of the two input image points.
  4. Calculate the affine transform between every triangle in the input images and the destination image ( morphed image ).
  5. Warp triangle from each input images to the output image, and blend the pixels based on alpha. Do this for every triangle and you get the output morphed image.

Hope this helps.

enter image description here

Satya Mallick
  • 2,807
  • 1
  • 20
  • 14
1

I don't know any libraries that do this specifically, but you could cobble together something yourself. You'd need a set of common fiducial points that you reference in all faces. Then you'd want to use those point to do Delaunay triangulation on it.

Now you can either do the transform directly from one face chassis to the other, or you can write it to an intermediary normalized face, make changes to that and then write it anywhere.

Community
  • 1
  • 1
wbest
  • 611
  • 1
  • 6
  • 15
1

Here are the steps of the face morphing implementation with mesh-warping algorithm (you could implement it with opencv or python scipy / scikit-image):

  1. Defining Correspondences: find point-correspondences between the faces to be aligned using facial landmarks (detect landmarks with dlib, e.g.,).
  2. Delaunay Triangulation: You need to provide a triangulation (Delaunay triangulation, e.g.) of these points that will be used for morphing (with scipy.spatial's Delaunay, e.g.,).
  3. Computing the Mid-way (morphed) Face: computing the average shape, warp both faces into that shape (calculate the affine transforms using the triangles in source image and the corresponding ones using the morphed image and warp the points inside the triangles and alpha-blend the warped images to obtain the final morphed image, using scikit-image's warp(), e.g.,).

The next animation shows the animated output from an implementation of mesh-warping algorithm with scipy / numpy / scikit-image in python (sequence of morph images starting from Monalisa image to Leonardo da Vinci image). This can be found here too.

enter image description here

Another popular algorithm is Beier-neely morphing algorithm (https://en.wikipedia.org/wiki/Beier%E2%80%93Neely_morphing_algorithm)

Sandipan Dey
  • 21,482
  • 2
  • 51
  • 63
0

Check a face-morphing tool in Python using OpenCV

Slava Litvinov
  • 392
  • 3
  • 7