CS194-26 FACE MORPHING

CS194-26 FACE MORPHING

Chendi Wu

Home Morph Sequence Population Caricature Smile and Makeup Class Morph Music Video
Defining Correspondences

The first step to create a face morphing sequence is to define pairs of correspondences between the images. These correspondences should pinpoint the most significant features that morphing bases on. Here, I selected 45 pairs of correspondences, denoted by green dots, for the photos of George and me.

Original George
Original me
George with points picked
Me with points picked
Computing the "Mid-Way Face" and morphing sequence

To create any intermediary face I along the morphing sequence from the starting image S to the destination image D, the following steps are performed:

  1. Calculate the target correspondences in the intermediary face by taking a weighted average of S and D's point coordinates.
  2. Use Delaunay Triangulation method to find a triangulation of the points on S, D and I.
  3. For each triangle, we compute affine transformation matrices T/T' mapping triangle coordinates in S/D to those in I. Then inverse warping and bilinear interpolation are used to find the corresponding pixel values from S and D, which are then weighted by a dissolving fraction to become the pixel values in the triangle in intermediary image I.
As an example, the "mid-way face" is presented here, which captures 1/2 of the shape and the color of both images. On the right of the "mid-way face" are the morphing sequence videos consisting of 45 intermediary morphed faces (including the original starting and destination images).

Midway from George to me
Midway from George to me
Another pair I played with
Her name is Zhu Yin
The "Mean Face" of a Population

In this part, I created the mean face of Brazillian researchers from the pre-labeled dataset: https://fei.edu.br/~cet/facedatabase.html. It contains 200 faces with 46 labeled points, among which 100 faces are emotionless and 100 are smiling. I chose only the smiling set of faces, and manually added points to the four corners of each image. To create the mean face, I first compute the target correspondences by averaging all points of the 100 images, and then repeat the process from the previous part to average the shape and color of each face.

Here are some sample faces from thee dataset (left), and the resulting images of them warped to the average correspondences (right):

Original
Warped to avg
Original
Warped to avg

Here is the mean face of all 100 smiling Brazilian researchers, which is just a nice smiley face with no prominent features (strong personal features are averaged out!):

"Mean face" of Brazilian researchers

Here are two experiments of morphing the "mean population" face to the geometry of my face and morphing my face to the geometry of the "mean population" face. The shapes are well-kept, but results are hilarious. Guess I just don't look like a Brazilian:)

Mean population to me
Me to mean population (Oh)
Caricatures

Subtracting the mean face features from our face features capture the prominent features of our faces. Caricatures can be made in this way. Hence, different from previous parts, the targeting correspondences are calculated using the formula: target_pts = mean_pts + alpha * (my_pts - mean_pts), alpha being a value smaller than 0 or larger than 1. Then the same morphing technique is applied to triangles, mapping the shape and color to the target images (caricatures). I created two caricatures using alpha = -1 and alpha = 1.5:

Me again
Alpha = -1
Alpha = 1.5
Bells & Whistles: Smile and Makeup

In this part, I morphed Michael Fassbender's stern face to my smiling face. Michael smiled when only morphing the shape, and he got makeup and a more feminine look when morphing both the shape and color!

Me again
Michael Fassbender
Just shape (smiling shark)
Shape and appearance
Bells & Whistles: Class Morph Music Video

I participated in the music video with some other classmates and here is the link to the whole video: https://youtu.be/O3vouduLS3w

Also, here is my part from it:

Me morphed to Bunny
Conclusion

This is a cool project! I got to make my friend a birthday video using her photos over ten years using the techniques from this project :) It's one of the best things in life to have a friend for ten years, and even better to have her for the rest of my life (I hope). By the time I submit this project, her birthday has already passed, but still, happy belated birthday!