Project 3: Face Morphing¶

Harish Palani (CS 194-26)

1.   Defining Correspondences

For the first component of this project, I used the faces of famed athletes Stephen Curry and Lionel Messi as captured by Martin Schoeller. Due to the symmetry between the images, four keypoints + four corners proved sufficient for alignment, with the resulting Delaunay triangulations shown below.

2.   Computing the "Midway Face"

Once the keypoints and resulting Delaunay triangulation were set, I computed the so-called "mid-way face" between the two images. I did so by first averaging the keypoints to yield a mean facial geometry before warping the two faces and subsequently averaging colors to produce the results shown below.

3.   The Morph Sequence

I then compiled this logic to derive a final morphing algorithm, yielding the animation below. The two images are first warped into the midway geometry before being cross-disolved, with interpolation of fractional weights enabling the seamless morphing patterns between frames as shown.

This morph's quality could be improved further by using a more extensive set of keypoints to generate the Delaunay triangulation core to the algorithm. For example, consider the set of points defining Stephen Curry's features below. This mirrors the IMM annotation sequence used in the Danes dataset, which will be explored further in the next section.

4.   The "Mean Face" of a Population

Using the IMM Face Database as mentioned above, I first computed the average faces of males and females, respectively, as annotated in the dataset. Images were not distinguished based on emotion — all neutral and happy faces were included, yielding the results shown below.

I then selected one random male and one random female and morphed each to the average geometry for their respective genders. The male had a neutral expression while the female was happy, adding to the diversity and intrigue of the outputs.

For the final component of this section, I selected a random male from the dataset to use in lieu of my own face. The original image as well as its provided keypoints are shown below.

This face was warped into the average male geometry and vice versa. As you can see, the latter output isn't of the greatest quality, as it's already a unified representation of several different faces prior to warping.

5.   Caricatures: Extrapolating from the Mean

Building directly on the outputs from the previous section, I modified the warp function to support fractional weights and subsequently created a morph between the random face selected above and the population mean. The resulting caricatures at various weights are shown in animated form below.

6.   Bells & Whistles

For bells & whistles, I created a morphing music video of some of Martin Schoeller's most celebrated portraits, containing transformations between a number of the world's most famous celebrities. The tune in the backdrop is the instrumental version of Still D.R.E. — a timeless classic with a piano beat that many have heard, but not all can identify in the present day.

For convenience purposes, four keypoints + four corners were used for Delaunay triangulation once again to yield reasonable results. Higher-quality morphing could be achieved with a more exhausive set of points similar to those provided in the IMM Face Database above.