Face Morphing and Modeling a Photo Collection

CS 194-26 Project 3

Monica Tang


Face Morphing

The goal is to create an animation that morphs one face into another.
A morph consists of two parts: a warp of the image shape and the cross-dissolve of the image colors.

Throughout the morph animation, we want the facial features to match well, so we need to manually define pairs of corresponding points. These points define the image shape. Then, the Delaunay triangulation on the midway shape is computed so that we can warp the images. The figures below illustrate the image shapes for the portraits of Anne Hathaway and Amy Adams. I used 61 points for each.

Photos taken by Martin Schoeller

In order to warp the triangles, we apply an affine transformation to each triangle:
Rather than forward warping (where we apply the transformation on the source triangles to send each pixel to its corresponding location), it is better to inverse warp (where we apply the inverse transformation on the destination triangles to grab the source pixel colors). This is because inverse warping eliminates holes and also prevents over- or under-computing, but one small issue arises as a result of using this method. After applying the inverse transformation on a pixel location in the destination triangle, the output may contain fractional values. In other words, it may land between pixels in the source triangle. So, we will need a function to interpolate the pixel color from its neighbors.

Then, we cross-dissolve the two images.

We repeat this process for each frame of the animation at increasing amounts of warp and cross-dissolve to produce a sequence of images. Below is the computed midway face as well as the final morph animation.

Here is another example of morphing using the faces of Mr. and Mrs. Andrews, from the Rococo painting Mr and Mrs Andrews by Thomas Gainsborough.

from Mr and Mrs Andrews by Thomas Gainsborough (1750)

"Mean Face" of a Population

Using a set of images of a population, we can compute the average face for that population. This can be done for subsets of a population, such as old, young, male, female, by ethnicity, etc.

Here, I used the Danes IMM Face database as my population. This dataset consists of 40 human faces, 7 female and 33 male. To compute the average face, we can simply compute the average shape, morph each image into the average shape, and cross-dissolve all of them.

Below are several of the dataset faces that were warped into the average shape.

We can also warp faces outside the dataset to the mean Danes face geometry, and also the Danes face to other face geometries. Below are examples using my face.

Caricatures

Previously, our warps used warp-fraction values (or alphas) between 0 and 1. But these are not the only possible values. We can extrapolate, and use alphas less than 0 or greater than 1 to produces caricatures. The following are examples of my face extrapolated from the Danes mean face using several different alphas.

Bells & Whistles

Additional examples using the above methods:

The following is a morph sequence of the members of a Korean/Chinese music group, called NCT, in order from oldest to youngest. There are 23 members (as of October 2021).

Below is the average face of all 23 members...resulting in the 24th member!