Project 4: Face Morphing

Brandon Huang, Fall 2017


The Midway Face

For the target of my face morph, I chose a picture of our very own Professor Efros. This face morph is based on the concept of the affine transformation and Delaunay triangulations. Essentially the idea is to break down two images into corresponding triangles, compute the average face shape, and morph both images into that shape before cross-dissolving them into an image that is midway in both shape and coloration.
Several steps below:

  1. I manually annotated corresponding points (tips of noses, jawlines, etc) on the two images to morph, and added correspondences for the corners of the two images so the whole image would be warped.
  2. I computed a Delaunay triangulation on the average of these two ordered lists of points. The average of the points provides a sense of the "average shape" of the two faces. The Delaunay triangulation is advantageous because it computes "good" triangles by minimizing the amount of long, thin triangles. This keeps the morph from becoming too distorted.
  3. For each of the two source images, I computed a affine transformation from each triangle in the average shape to the corresponding triangle in that source image. I did this by solving a system of linear equations to recover the 6 parameters of the affine transformation matrix. The 3 (x,y) pairs that make up each of the triangles provides enough information to fully determine that system of linear equations.
  4. I computed an inverse warp from each of the source images to the average shape, by transforming each pixel of the average shape and interpolating its value from the source image. This produces two images warped into the average shape.
  5. I cross-dissolved the two warped images to produce a final midway face.

Source image: my face. I tried to emulate the professor's face angle and expression.
Source image. Hi Professor!
Blended image (halfway shape and cross-dissolve)

Morph sequence

By applying this morph with varying contributions of shape and color from each image, we can animate a quite smooth morph between the two faces.

Spooky...

Mean face

Using an annotated dataset of faces, we can compute a "mean face" that averages not only color but shape. I downloaded a dataset of Danish folks and, using a similar procedure as above, computed an average shape using the annotated points. I then warped every face into the average shape and averaged the pixel values.

The dataset contains considerably more male examples so there is a clear bias toward male facial features. Interestingly, the face is fairly aesthetically pleasing. This might be because humans find more symmetric features attractive, and the averaging of samples smooths out the individual asymmetries of each person.
Took a new picture of myself, more closely aligned with the facial angle of the Danes dataset. I annotated this to match the annotation of the Danes dataset, which differs from the annotations I did on mine and Efros' faces.
Result of morphing my face into the average shape of the Danes dataset. Looks like the folks in the dataset have chubbier faces than mine, on average.
Result of morphing the average of the Danes dataset into my geometry.

Caricature

A "caricature" of my face can be produced using another face as reference. Instead of morphing toward the average, which is 0.5 * face1_shape + 0.5 * face2_shape, we can instead extrapolate farther in one particular direction, for example 1.5 * face1_shape - 0.5 * face2_shape. This produces a face which can be said to have "50% more of whatever distinguishes me from the average Dane".

Original.
1.3 * original - 0.3 * danes
1.8 * original - 0.8 * danes
2.3 * original - 1.3 * danes

Reference

M. B. Stegmann, B. K. Ersbøll, and R. Larsen. FAME – a flexible appearance modelling environment. IEEE Trans. on Medical Imaging, 22(10):1319–1331, 2003