CS 194-26 Project 3: Face Morphing

Eric Li

Overview

The goal of this project was to use linear algebra to warp different faces. There were two parameters, the amount of warp that we wanted to use, as well as the dissolve fraction. First, we needed to define correspondences between the points in different images. This was done by defining a set of points that corresponded to the same features in both images. We then used Delaunay triangulation to generate triangles for the points of the mesh we have created.

Here are the two subjects we will be morphing today.

Here are their triangulated faces.

Computing the "Mid-way Face"

Here, we will use an affine transformation to warp the two faces to the "mid-way face". We do this by computing the average of the corresponding points in each of the faces so we know the coordinates of the goal of our transformation. For each triangle in the "mid-way face," we compute the inverse transform in order to get the pixel value at that point. This is accomplished through two inverse transforms to the first and second face pictures respectively. We also cross dissolve by multiplying the image intensities for pixel intensity values that we retrieve from the original image by one half. When we add these two images, we get our "mid-way face".

Morph Sequence

We recalcuate the target image coordinates by taking a weighted average of the two image coordinates by alpha and (1-alpha). We also weight our image intensities by alpha and (1-alpha) respectively. Adding these two images together after weighting them and morphing them to the same target will give us the composite image for every alpha level.

The "Mean Face" of a Population

We now did what we did above sequentially for a large set of images. We first calculate the mean coordantes for the entire image set by the coordinates over the entire image set and dividing the summed coordinates by the number of images. After this, we calculate the "mean face" by warping every face to these new coordinates and taking the average of the matrices.

Here are some examples of specific subjects that I used in my population set.

And here are their faces warped to the shape of the mean.

We warp our picture of Elon Musk to the facial structure of the mean face of the Brazilian set.

We warp our picture of the mean Brazilinaa face to the facial structure of Elon Musk.

Caricatures: Extrapolating from the mean

We can warp our face by extrapolating from the population mean. We can play with the alpha values in order to get a more "extreme" face for Elon Musk. His facial features that differ from the reference dataset are exaggereated when we create caricatures at increased warp levels. The alpha values here are linearly interpolated between 1 and 1.5, where an alpha of greater than 1 denotes an abnormal "extreme" warp.

Bells and Whisles: Changing Elon's Age and Expression

We can set the target structure for our warps to be average images with different characteristics. By finding a mesh that captures the structure of a face with different characteristics, we can change different things about Musk.

Let's make Elon frown.

This is an example of frowning Elon Musk at different levels of appearance bleed, where alpha = 0.3, 0.5, 0.7.

We can also make Elon Musk a baby.

This is an example of baby Elon Musk at different levels of appearance bleed, where alpha = 0.3, 0.5, 0.7.

Let's do an extreme extreme of warping. I'm not using the population mean for this example because I couldn't find a population average for such an extreme expression. However, we can use this famous picture of Obama as a reference for facial structure. Here's Elon Musk's face with Obama's facial structure and laugh.

We let Obama's appearance bleed through with alpha = 0.3, 0.5, 0.7.

Bells and Whisles: Morphing the Burj Khalifa through Time

This is a music video showing the Burj being warped to different times of day and artistic styles.