CS 194-26 Computational Photography Fall 2018
Guowei Yang cs194-26-acg
Human face is an interesting subject to manipulate with. With modern techniques, we could easily perform some "morphing" operation on faces and generate some really cool transformations. In this project, we will play around with human faces by manipulating the pixels and their relative locations, and obtain some morphing sequences.
Part 1: Face Morphing
To morph two faces, we have to perform several operations:
Step 1: Define Correspondences
In order to have a successful morphing, we need to define correspondences for the two input images. We want to have eyes morphing into eyes, nose morphing into nose, mouth morphing into mouth, etc. Therefore, we have to define key points so that each element in the left image corresponds to the same element in the right image, which is shown below:
Correspondences of Professor Hug & DeNero's face
Step 2: Compute the "Midway" Shape
After defining the correspondances, we want to compute a Delaunay Triangulation of the points. In practice, it's much better to use the Delaunay Triangulation of the "midway shape" of the two input points, i.e. the midpoint of each pair of corresponding points on the faces. We will use the Delaunay Triangulation on the midway face as the triangulation for the two source images.
Calculate the midway point of each corresponding pair of points
The midpoints of each corresponding points
Step 3: Affine Transformation of Input Images
After finding the triangulations, we want to fill in the pixels. In order to do that, we have to find the affine transformation matrix for each triangle, and use the inverse affine transformation to find corresponding pixel values for each (x, y) entry in the midway image.
Image A Triangle Points
Image B Triangles
Finding corresponding pixel values to fill in midway face by using inverse affine transformation
Step 4: Construct the Morphing Sequence Frame by Frame
Now we know how to go from source image to the midway. To make an entire morphing sequence, we basically increment the average weight from 0 to 1 in 45 steps to create a 45-frame sequence. Once we obtain the sequence, simply replay the sequence in 30fps, and voila!
What about morphing myself?
Robert Downey Jr.
Robert Downey Myself
Part 2: Mean Face of Population
Now we know how to find an average of two faces. What will happen if we average a bunch of faces of different people drawn from the population?
Step 1: Obtain the Dataset
There are multiple resources for finding facial dataset. The one I picked is the FEI Face Database.
Step 2: Averaging
Similar to finding te midway face, we simply perform an average on the corresponding points and to obtain a mean shape, and then we borrow the pixel values from all the images in the database and perform an average again.
Average face of 100 people
Average face of 100 people Similing
Step 3: Messing Around
After obtaining the average, we can play around with it, we could morph the input dataset to the mean face, or myself to the mean face.
Average of myself and the mean face
Averaging Sequence of the dataset
Part 3: Caricature
Using the result of the mean face, I could use that information to enhance the features on my own face. This could be easily done by changing the weight to negative:
Part 4: Bells and Whistles
We have created a small morphing sequence of around 20 people :)