CS 194-26 Project 3: Face Morphing

Catherine Lu, Fall 2020

Overview

In this project, I produced various images involving morphs of faces, including a morph between two different faces, a poplulation "mean face", a caricature of my own face, and transformed the expression of a face.

Face Morphing

Defining Correspondences

The first step in morphing two faces together is picking points on each face that correspond to each other in order to morph the shape of one face into the other. From the points selected, I also computed a Delaunay triangulation of the average face between the two and applied it to each set of points.

Computing the Mid-Way Face

To compute the mid-way face, I needed to compute the average shape, then warp each of the faces to that shape and averaging the colors of the two faces onto that shape.

For the average shape, I averaged the correspondence points of the two images. I used the Delaunay triangulation to segment the image into triangles and computed an affine transformation for each triangle in each of the two images to the triangle in the average shape. To compute the matrix corresponding to this affine transformation, I solved a system of linear equations from the vertices of one triangle to the vertices of the average triangle.

In order to properly warp the images, I performed an inverse warp where I mapped the destination triangle, i.e. the triangle from the average image, to each of the two images I wanted to warp. Then, I obtained the colors for that triangle from the original images by interpolating from the original pixel values. Finally, I generated the end result by averaging the interpolated colors from the two images.

The Morph Sequence

Using the method from creating the mid-way face, I was able to create a sequence of morphs by changing the fraction of each image that was included in the shape and color.

The "Mean Face of a Population

The idea of morphing faces can also be applied to computing the average face of a population. Here I took around 200 images from a Brazilian face database that was already labeled and computed the average shape of these faces by averaging the annotated correspondence points. Then, I morphed each image to the average shape, then averaged the colors of the images.

Here I have shown my face warped to the shape of the average face in the first image and the average face warped to the shape of my face in the second image.

Caricatures

One fun application of these mean faces is producing caricatures. The idea behind cartoon caricatures is that the artist will emphasize characteristics of a face that stand out. This can be produced computationally by subtracting the mean face from a particular face, which results in only outstanding characteristics in that difference image. These differences can be added back to the original image to exaggerate them.

Bells and Whistles

Here I have changed the expresssion on my face from a smiling expression to a more serious one. I used the images from the Brazilian dataset which were categorized into smiling and frowing images. I computed an average over the sad faces and subtracted the mean face from the average sad face, giving only "sad" characteristics. Then I added these sad characteristics to my own face. Here we can see what this looks like with changing just the shape, just the appearance, and then morphing both.

Note: The dataset images were in grayscale, so when morphing appearance I could only produce a grayscale photo