CS194-026 Project 4: Face Morphing

Franklin Heng



Project Introduction

In this project, we introduce facial morphing. This technique and visual effect is seen in many applications ranging from music videos to modern day mobile phone apps. In order to accomplish this animation, we will take advantage of computer graphics concepts such as triangulations, to smoothly interpolate colors and gradually transform image pixels.


Part 1: Computing Mid-Way Face

Computing the Mid-Way face between two images is essential and core to face morphing. In order to get the average face, we executed the following steps:

  1. Manually select matching feature points in image1 and image 2. In the images below, we have selected about 40 pixel coordinates for both images.
  2. Using these two pixel coordinate sets, we compute an set of pixels that are the average of the two. While the selected matching features may differ when morphing between other images we will always select the four corners of the image.
  3. Once we have the average pixels, we will triangulate all three coordinate sets (two original that we manually selected and one average). For my examples, I used Delaunay Triangulation.
  4. Then for each triangle (formed from manually selected points), in each image, we will compute the affine transformation that transforms to the respective average triangle, using linear algebra.
  5. Once we have all the affine transformations, we will do an inverse affine transformation to map the source image to the designated average image, and use linear interopolation to extract the colors.

Mathew McConaughey & George Clooney Average:


Me & Glenn from Walking Dead (RIP) Average:


Part 2: The Morph Sequence

For this part of the project, we will use the same algorithm from part 1, to create a sequences that morphs one face to another. The process is very similar to computing the average face. However, this time we add in two parameters dissolve fraction and warp fraction. The warp fraction is used to define how much we average ther feature points (x, y pixel coordinates) and ther dissolve fraction is the weight used for interpolating between both images.

Mathew & George Sequence:


Me & Glenn Sequence:


Part 3: The "Mean face" of a population

To create a mean face of a population, I used Danes IMM Face Database which consisted of 240 face images. In order to create the mean face, we follow the same procedure mentioned above, where we average the selected points, traingulate the points, and compute and apply the affine transformations. For these images, the average will be the size of the image dataset (240), and we apply this to both averaging the coordinates and interpolation.

Example Image from Dataset

Mean Face of Population

Example Image from Dataset


Part 4: Caricatures: Extrapolating from the mean

In order to exprapolate features from the mean, we follow the steps:

  1. Collect correspoding points between my face and the average population face, and calculate the average points from both.
  2. Compute triangulation of average points.
  3. Compute the difference from my points from the average points
  4. Sum together my points with the difference * weight (where weight is a value that we vary [0, 1])
  5. Lastly, warp the image as usual, with the final sum as computed above as the destination triangle.

Intuitively, by subtracting my points from the average, we are extracting the differences or areas of my face that are different than the mean. Thus, as my weight goes from 0 to 1, I am applying more emphasis on my differences. From comparing my original image to the mean population face, we can note three things: I have a bigger nose, bigger smile/teeth, and thinner cheeks because I am smiling more. Thus, as I increase the weight variable, we should notice my nose, teeth, and smile getting bigger, and my cheeks/jaw getting thinner.

weight = 0

weight = .5

weight = 1.0

weight = 2.0


Part 5: Bells and Whistles

For this bells and whistle I chose to change gender by morphing myself with an average female that would look closest to me (average Cambodian female).


Conclusion

In conclusion, I have confirmed to myself emperically that Mathew McConaughey can morph with anyone and still look good. Jokes aside, I found the methods to face morphing pretty interesting and the project itself fun.