Image Manipulation and Computational Photography

Project 4: Face Morphing

Jianglai Zhang

Overview

This time we explore how to produce smooth morphing between images, specifically with human faces. With image morphing, we could produce morph sequences / videos between two images, calcuate the "mean" face of a group of people, "change" the gender of our image subject, and to create carricatures using morphing and extrapolation.

Define Correspondences

First thing first, we need to define a set of pairs of points representing the correspondences between key feature points in each of the two images where we are about to morph each into the other. I first picked two photos that are approximately of the same size, with the subject in the center of each, align and resize the photos, and then selected 34 points where I think are key features of human faces. Then I pick the average of each pair of the points and use the average coordinates to do a Delaunay triangulation. Here are the key feature point selections for me and my friend George as well as the Delaunay triangulation for the mid-way coordinates:

The Mid-Way Face

After defining the key feature points correspondences, calculating the average coordinates and use it to create a Delaunay triangulation, we get the mid-way triangles. Now we simply warp each triangle into the corresponding triangle in the source image as well as the destination image's corresponding triangle. This step is basically an affine transformation for each triangle. In addition to the shape warp, we also do an average on the appearances or pixel values of the two images. And then we get a "mid-way" face:

Morph Sequence

Then we can create a morph sequence by setting different warp factor and dissolve factor for the morph process. Here I used step size 1/45, which gives us 45 frames in the morph process between the two images:

The Mean Face of a Population

Now we can also compute the "mean" face for a population. Here I used the FEI Face Database's first 100 spatially normalized images in which the subject is smiling since they are very well annotated (key feature points). And then we can morph each of the actual image in the set to this population mean's shape, or maybe also morph my own image to this mean shape. Note that the images provided are in grayscale and that the annotation does not care about areas outside the face and thus I simply make the rest of the area black:

Caricatures

We can also create caricatures using morphing and the mean face we got. This is done by simply taking the other direction and extrapolate the feature differences of my own image from the population mean we got. Here is a caricature of me with alpha = 1.8:

Bells and Whistles

With the same method, we can try to "change" the gender of an image subject. I first picked out some images with female subjects and calculated the "mean" face of this group, and then morphed my own image to 1) only the shape of the mean and 2) both the shape and appearance of the mean. Here are the more female (?) versions of me:

Credits

Inspiration taken from friends, past projects, python libraries, papers and other online resources.