Face Morphing

Jingwei Kang, cs194-26-abr

Overview

Part 1: Face Morphing

In this part of the project, I learned to create a "morph" animation between faces, which entails a warp of image shape as well as a cross-dissolve of color. To warp the shape, we first had to align the images and select points of correspondence (using cpselect()). Next, I generated a Delaunay triangulation from the average points of correspondence, and used linear algebra to find the affine transformations between corresponding triangles in the original and warped iamges. Lastly, for each corresponding triangle, I used inverse warping to populate the pixels of the warped image from pixels of the original image.

Part 1.1: "Mid-Way Face"

As a tribute to Kobe Bryant, I wanted to morph between past and present Laker superstars. This morph worked well on the face but poorly on the outfits. This is expected because the majority of my correspondences were points on the face and they were wearing very different outfits. Furthermore, we've seen that features in faces are generally continuous, but the same cannot be said about clothing.

Part 1.2: Morph Sequence

The midway face above corresponds to a midway warp (structure) and cross-dissolve (appearance) of the two images. It was then pretty straightforward to create an array of 45 linearly spaced weights between 0-1 to create a smooth transition between the structure and appearance of the 2 faces.

Part 2: "Mean Face" of Population

For this part, I utilized the Brazilian FEI face database, which came with 200 prelabeled faces with 2 expressions each.

Part 2.1: Compute Average Shape

Since the faces were prelabeled with 46 points each, it was straightforward to extract these from the .pts files and average them. I used the entire population for this.

Part 2.2: Morphing into Average Shape

Below I show a couple examples of morphing each face into the average shape of the data set. The first two examples show a face that is wider than the average, the next two show a face that is longer than the average, and the last two show a face that is closer to the average shape.

Part 2.3: Average Brazilian Face and my Face

After morphing each face into the average shape, it was easy to compute the average face (appearance) by averaging all the images.

I also morphed my face into the average Brazilian geometry and vice versa.

Part 3: Caricatures

To create a caricature, I can morph my face closer or farther from the population mean I calculated. I used a weight "alpha" to determine the structure I would morph my face into (alpha=0 was my structure, alpha=1 was the structure of the population mean).

Part 4: Bells and Whistles

Part 4.1: Change Gender and Ethnicity

I was curious how I would look as a German woman.

Part 4.2: Morphing Music Video

I created a music video (use this link if embed doesn't work) of the Red Mill (Clinton, NJ) over the seasons. Unfortunately, it was difficult to use the code provided to align multiple images (and retain the exact same size), so the Red Mill does move throughout the frames. The song used is "Autumn Leaves" by Eric Clapton.