CS194-26 Project 4: Face Morphing

Josh Zeitsoff, cs194-26-abi


For this project, our goal was to morph one face into another. The first thing we did was to define correspondences, or equivalent points in 2 images. For example, mapping a mouth in the source image to a mouth in the target image. We did this by starting with 2 images of the same size, and selecting 20 points per image. Most of the correspondence points are around the face, in an attempt to accurately match fine grained facial features such as lips, teeth, and eyes. In addition, I added 4 correspondence points at the corner of the image to keep the background.

Source Image

Annotated Source Image

Target Image

Annotated Target Image

Midway Face

To compute the midway face, I took the set of correspondence points from both the source and target images and averaged them. Then, I warped each image into the average face shape. To warp each face towards the average face shape, I triangulated the points for the average face shape using a Delaunay triangulation. Each triangle vertex in the midway face shape corresponded to a point in the original source or target image, so I had a mapping of triangles in the midway face to triangles in the source image and the target image. Next, I went through each of these triangles and computed an affine transformation to map pixel locations in the midway shape back to pixel locations in the source or target image.

Computing the affine transformation involved solving a matrix of this form for each pair of triangle vertices, where the x' and y' coordinates come from the midway shape and the x and y coordinates come from the source image. Thus, I had to solve 6 total equations, with 6 unknowns, since there are 2 equations per vertex and 3 vertices per triangle.

After solving this matrix, I took the inverse, since the eventual goal was to map coordinates in the midway face back to coordinates in the source or target image and then select the pixels at those coordinates. In the event that they did not exactly map back to pixels in the source or target image, I used interpolation to determine what the pixel values should be. Now that I had matching coordinates in the midway face, source image, and target image, as well as the pixel values, I equally weighed the values from the source and target image to form the midway face, seen below.

Midway image

Morph Sequence

For the actual morphing of one the source image into the target image, I had to repeat the process above but with varying weights that selected more from the target image and less from the source image over the course of 45 timesteps.

George Taylor Gif

Josh Soph Gif

Mean Face of a Population

For this part, I used the FEI "frontalimages_spatially_normalized_part1" and "frontalimages_spatially_normalized_part2" image datasets, as well as their corresponding points. To find the average face shape, I just averaged all of the points equally to achieve a set of points. To morph each face into the average face shape, I repeated the process to compute the midway face, but selected pixels only from the source image. Thus, I essentially mapped each triangle in the average face shape back to its corresponding triangle in the source face shape, then selected pixels from the source face shape.

Faces Morphed into the Average Face Shape

Ex1: Face Morphed Into the Average

Hover over to see the face morphed into the average

Ex2: Face Morphed Into the Average

Hover over to see the face morphed into the average

Average Face of Population

To compute the average face of our population, I used the images that I had morphed towards the average face and weighted each equally so that the average face consisted of equal pixel amounts from each image.