Overview

Faces can be represented by a set of keypoints that correspond to distinctive features (e.g. nose, mouth, eyes, etc.) as well as the color values of the face itself. We thus can perform mathematical operations on set of these faces and do cool things, such as averaging across the locations of keypoints as well as the color values to create a face that is a hybrid of two people.

Defining Correspondences

In order to get a good result, correspondences (keypoints) must be defined first. This ensures a variation in facial structure will not create a blur between two faces, as the keypoints themselves will also be warped in the hybrid model. In this project, I used my own face, as well as my roommate Leon's. Both are shown below.

We also need a way to connect these keypoints in a planar graph, so that we can determine which patches of the face should correspond color-wise to each other. I used the Delaunay triangulation on my own face to determine this. The correspondences between the two faces are shown below.

Computing the midway face

To compute the midway face, we first can determine the facial structure by simply performing an average of the two sets of waypoints. This is as simple as taking each pair of corresponding vertices and taking a literal average to find the new vertex coordinates.

However, we also need to figure out what colors will go into each of our triangles. This can be done by computing the affine matrix transformation between two triangles. We can then perform an inverse warp by looking at each pixel within our triangle of interest on our desired end face, and multiplying its coordinates by the inverse of the matrix transformation to get the coordinates of which pixel in our original image our current pixel corresponds to. We can do a simple interpolation if this pixel does not land on integer coordinates.

After performing this for all triangles, we get the end result shown below.

The Delaunay triangulations for each image are also shown below.

In order to generate a smooth transition between the two faces, we can simply just slowly vary the weighting of each face, and compute a biased average in that way. We have two variables to control here: the weighting of the face shape, and the weighting of the face coloring. I was able to obtain a good video result with just linearly varying each one equivalently across the 45 frames I rendered.

Population Means

We can also compute the mean of a population provided a dataset, and morph any individual to fit the characteristic of the mean population. The mean of the Brazilian dataset is shown below, along with some individuals (myself included) warped to fit the average face shape.

The opposite can be done too, warping the average to conform to my own face geometry:

Caricature

Provided an average and a datapoint (me), we can extrapolate past the datapoint by setting a negative value for the face structure weighting in the face hybridization function. I simply reused the average face I calculated from the past part; the results are shown below.

A single Bell (or a single Whistle)

I attempted to modify my face image to look like the other gender. I found an average east Asian female face online, and modified my face's geometry and shading to match it.