Face Morphing

CS 194-26 Computational Photography Fall 2018

Guowei Yang  cs194-26-acg 

Human face is an interesting subject to manipulate with. With modern techniques, we could easily perform some "morphing" operation on faces and generate some really cool transformations. In this project, we will play around with human faces by manipulating the pixels and their relative locations, and obtain some morphing sequences.

Introduction

Part 1: Face Morphing

To morph two faces, we have to perform several operations:

Step 1: Define Correspondences

In order to have a successful morphing, we need to define correspondences for the two input images. We want to have eyes morphing into eyes, nose morphing into nose, mouth morphing into mouth, etc. Therefore, we have to define key points so that each element in the left image corresponds to the same element in the right image, which is shown below:

Correspondences of Professor Hug & DeNero's face

Step 2: Compute the "Midway" Shape

After defining the correspondances, we want to compute a Delaunay Triangulation of the points. In practice, it's much better to use the Delaunay Triangulation of the "midway shape" of the two input points, i.e. the midpoint of each pair of corresponding points on the faces. We will use the Delaunay Triangulation on the midway face as the triangulation for the two source images.

Delaunay Triangulation

Calculate the midway point of each corresponding pair of points

The midpoints of each corresponding points

+

Step 3: Affine Transformation of Input Images

After finding the triangulations, we want to fill in the pixels. In order to do that, we have to find the affine transformation matrix for each triangle, and use the inverse affine transformation to find corresponding pixel values for each (x, y) entry in the midway image.

Image A Triangle Points

Affine Matrix

Image B Triangles

Finding corresponding pixel values to fill in midway face by using inverse affine transformation

Step 4: Construct the Morphing Sequence Frame by Frame

Now we know how to go from source image to the midway. To make an entire morphing sequence, we basically increment the average weight from 0 to 1 in 45 steps to create a 45-frame sequence. Once we obtain the sequence, simply replay the sequence in 30fps, and voila!

What about morphing myself?

Myself

Robert Downey Jr.

Robert Downey Myself

Part 2: Mean Face of Population

Now we know how to find an average of two faces. What will happen if we average a bunch of faces of different people drawn from the population?

Step 1: Obtain the Dataset

There are multiple resources for finding facial dataset. The one I picked is the FEI Face Database.

Step 2: Averaging

Similar to finding te midway face, we simply perform an average on the corresponding points and to obtain a mean shape, and then we borrow the pixel values from all the images in the database and perform an average again.

Average face of 100 people

Average face of 100 people Similing

Step 3: Messing Around

After obtaining the average, we can play around with it, we could morph the input dataset to the mean face, or myself to the mean face.

Average of myself and the mean face

Morphing Myself

Averaging Sequence of the dataset

Part 3: Caricature

Using the result of the mean face, I could use that information to enhance the features on my own face. This could be easily done by changing the weight to negative:

Part 4: Bells and Whistles

We have created a small morphing sequence of around 20 people :)