Overview

In this project, I morphed faces into each other by matching up the shape of the face through key points and then averaging the color from each original image together. We used triangulation of the key points to find the color for each pixel. We then produced the average Danish face and used that average to make a caricature. Finally, I used different average faces To change my age, gender, and race.

Defining Correspondences

To define correspondences, I chose a total of 77 key points between myself and my friend. These include important facial features, like eyes, nose, mouth, ears, and the hairline. I also included the glasses because we both wear glasses, so I want the glasses to morph well into each other. We triangulated these key points using Delaunay triangulation shown below.

Computing the "Mid-Way" Face

To create the "mid-way" face, I found the average shape by taking the average of each of the key points of the original images. I then found the triangulation of the average image. I used inverse warp to find the corresponding pixel values for the average image. After finding the affine transformation from each average triangle to the original triangle, I found the corresponding pixel values in the original images for each pixel in each triangle. I then used bilinear interpolation on each of those pixel values found to find what color the pixel in the average picture should be. I did this for both original images, and then averaged the pixel values found together to get the average face shown below.

Morph Sequence

For the morph sequence, I calculated the weighted average face shapes between my original picture and my friend's original picture, so that it was evenly divided into 45 frames total. For each of the intermediate frames, I calculated the weighted average by interpolating between the two points by a specific factor, based on which frame it was. I then found the points of the warped triangles for both pictures, and took the weighted average of those colors with the same factor as used when warping the triangles. I then put all of the frames together to get the final gif of the morph.

The "Mean Face" of a Population

I used the Dane dataset, and I used all of the pictures where people were facing forward to get an average Danish face. I followed similar steps to finding the "mid-way" face earlier by taking the average face shape of all of the Danes, and I found the transformation to go from the warped triangles to the original triangles for each image. I then found the corresponding pixel values for each warped triangle pixel through invere warping and bilinear interpolation. Finally, I averaged together all of the warped appearances to get the final average face.

Here is the final average Danish face.

Here below are the first 20 Danes in the dataset, warped to the average face shape.

I then warpped my own face into the geometry of the average Dane. I did this by cropping the average Dane picture to the same size as my own, and mapped them together with a new set of key points. Some of the warpping does not look good, like my glasses because the average Dane was not wearing any glasses, so it is not a subspace.

Additionally, I warped the average Dane face to my face shape with the same process of mapping these key points and using inverse warp to find the pixel values.

Caricatures: Extrapolating from the mean

To extrapolate from the mean, I found the difference of the key points between my face shape and the average Dane face shape. I generated a new face shape by adding this difference to the average Dane face by creating a caricature with less of those emphasized features and also more, depending on the multiplicative factor. Having a factor of 1 will just get back the original picture. The results are shown below, with their respective multiplicative factor.

B&W: Change gender/age/race

To change these attributes, I found the face of the average Chinese female, the average baby, and the average Korean male. I then mapped on the same key points onto these three average faces and my own face. To morph just the shape, I found the average shape between my face and the average faces I found on the internet. Then, I transformed my face shape to each of average shapes, and used invert warping to map the color to the correct pixels. To change the appearance, I morphed each of the averages to my face shape, and took the average of my original picture's pixels and the average face's pixels. Finally, to blend both the appearance and shape, I changed the shape of the average faces and my face to the average shape between the average face shape and my face shape. I then averaged together the pixel values for my face and the average face to blend both shape and appearance. The examples are shown below.