CS 194-26 Project 3:

Face Morphing

Leon Xu


1. Morph Sequence

We can morph images together by first averaging the shape, and then the color. Shape averaging is achieved by averaging the coordinates of a set of correspondence points, e.g. nose-to-nose, mouth-to-mouth, etc. These correspondence points are shown below, along with Delaunay triangulation:

Me
George Clooney

After averaging these correspondence point locations, we can perform a Delaunay triangulation, and then apply affine transformations on the triangles in the original images to transform them into the new shape. Note that there is some distortion due to my hair not matching Clooney's very well, as well as covering my eyebrow.

Me, warped to average shape
George Clooney, warped to average shape

The specific implementation actually uses the inverse transform, and finds the interpolated pixel values from the original image given the coordinates in the average shape. Then we cross-dissolve the colors (a.k.a. average the color values) to get a morphed image.

In addition, we can take a weighted average when warping and cross-dissolving to get a morphed image that is more similar to one or the other of the original images. By computing morphed images with weights of \((t, 1 - t)\) for \(t\) in range \([0, 1/44, 2/44, \dots, 43/44, 1]\), we can produce a 45-frame morphing video:

Me, morphing into George Clooney

Mean Face

We can expand upon the method above to calculate a mean face of a population. Using a dataset of Danish faces, choosing the male faces in the same pose (Pose 1: full frontal face, neutral expression, diffuse light), we average the correspondence points of all 33 faces to an average shape, warp each face into that average shape, and then cross-dissolve the pixel-by-pixel color values of all these warped faces.

Average Danish face

Some examples of faces warped into the average shape. Note that the foreheads sometimes don't fit well with the rest of the image: this is because the provided annotations don't include many points on the upper half of the face. I tried to mitigate this issue by adding points on the corners of the images as well.

Dane 0, warped to average shape
Dane 10, warped to average shape
Dane 20, warped to average shape
Dane 30, warped to average shape
Annotations for data set

In addition, my own face warped into the average shape, and the average face warped into my shape. Note that this uses my own annotation schema seen in the earlier morph sequence, rather than the schema from the Danish dataset.

My face with average Danish shape
Average Danish face with my shape
My face with average Danish shape, using their schema

Caricatures

We can also use this shape warping technique to create caricatures: consider the face shape vectors \(\vec u\) and \(\vec v\). Then we can define \(d = \vec u - \vec v\). Normally, to warp, we take the average face \(\vec v + \vec d / 2\). But we can overemphasize the effects of \(\vec d\) and instead compute the shape \(\vec v + 3 \vec d / 2\), to get a caricatured face.

Below, my face caricatured in the direction of the average Danish face.

My face with Danish caricature

Note that this is just a shape warp without color cross-dissolving.


Bells & Whistles

I also aligned my own face to an average Asian female face found on the internet, using the same correspondence point schema as in the morph sequence.

Average Asian female face
My face
My face with average Asian female shape