Project 4: Face Morphing

Author:Tianrui Guo <aab>

Project Spec:

Project Writeup

Project Overview

The purpose of this project is to morph from one face to another face. To accomplish this, we must warp the image shape and then cross dissolve the colors. The cross dissolve is handled by taking a weighted sum of the two images. To warp the image shape, we must first define a set of point correspondences on the two images, and then compute a consistent triangulation on the these points. Then, the problem reduces to simply warping each pair of triangles in the source images.

Face Morphing

Defining Correspondences

The first thing to do is define pairs of corresponding points on the two images by hand. While the rest of this project was written in Python, I chose to use the cpselect tool in Matlab to pick the points because it was easier to use. I then exported the points to a .mat file and loaded them into Python.

Affine Transformation

The next thing we need to do is to compute the affine transformation based off a source and target triangle. Let \(P_1 = \begin{bmatrix} x_1 & x_2 & x_3 \\ y_1 & y_2 & y_3 \\ 1 & 1 & 1 \end{bmatrix}\) be the matrix corresponding to the source triangle, where \((x_1, y_1)\), \((x_2, y_2)\), and \((x_3, y_3)\). We can define a matrix \(P_2\) in a similar manner for the target triangle. Then, we are interested in a transformation matrix \(T\) such that \(T P_1 = P_2\). Thus, we can compute \(T\) with:

\[T = P_2 P_1^{-1}\]

Note that for the face morphing problem, the source points are from the original images, and the target points are obtained using a weighted sum of the original points. We fill in the pixels in the target image by using the transformation matrix for the corresponding triangle to find the matching points in original images. Note that the transformation will result in non-integer indices in most cases, so we must do some linear interpolation to get the pixel values. I used scipy.interpolate.RegularGridInterpolator for this.

Morph Sequence

I chose to morph Jerome Powell and Janet Yellen, the current and former Fed chairs. Here are the results:

Morph gif
Midway Face

Another Example

Morphing between Scott Shenker and Taylor Swift:

_images/tayshenker_small.gif _images/tayshenker.jpg

Population Mean Face

Using the face morphing function, we can manipulate faces from certain datasets. For this project, I chose to use this dataset:

Mean Face

We can see what the average face looks like by finding the average shape and warping the faces to the average shape and then merging them.


Morph to Mean

Here are examples of individual faces being morphed to the mean:

_images/morph_to_mean_00.jpg _images/morph_to_mean_10.jpg _images/morph_to_mean_20.jpg _images/morph_to_mean_30.jpg

We can also morph Jerome Powell to the shape of the average Danish face, or vice-versa

_images/powell_to_mean.jpg _images/mean_to_powell.jpg


If we extrapolate from the morph, we can produce caricatures.


Bells and Whistles

Changing Gender

We can try to morph Jerome Powell to the average Danish female. We can morph the shape, appearance, or both.


Changing Gender, v2

Another thing we can do is to look at the difference between the average male and female faces, and then add that difference to the points on Jerome Powell’s face.

_images/men_mean.jpg _images/women_mean.jpg _images/fem_powell.jpg