Project 4: Face Morphing

Utkarsh Singhal

Introduction

The goal of this project is to use affine transforms and Delaunay triangulation to morph one face to another. The naive way to do this is to use cross-dissolve, which is a simple linear interpolation between the two images. However, the result looks unnatural and unappealing. A better approach is to interpolate geometry as well. To achieve this, I annotate keypoints on both images and compute the mid-way geometries by interpolating the keypoints linearly. Using affine transforms, I can morph the facial geometries to match the mid-way geometry, and blend the two warped images to create a mid-way frame. This allows for much smoother animations. On top of that, I also use a dataset of pre-annotated faces, and I show that the first 2 principal components in that data correspond to face orientation.

---

Computing the Mid-Way face

As described in the intro, I can compute midway geometry, and transform the initial and the target images to it before cross-dissolving them.

With triangular meshes

Morph Sequence

In accordance with the instructions, I created 45 midway frames and rendered an animation for this morphing sequence. I tried using non-linearly scaled ratios for cross-dissolving, but the effect was insignificant compared to other factors like proper alignment and annotation.

The "Mean Face" of a population

Without any alignment or morphing, the average looks quite blurry. But a clear face emerges once we actually morph the images accordingly

It is interesting to see that the average shape is relatively thin compared to some faces in the dataset. My guess is that this happens because we are averaging over faces in different orientations: left, right, center, and having all points closer to the center helps the optimization.

This effect becomes much more clear when my face is morphed into the average geometry. Even though my face is very compatible with one of the examples shown above, it is not as compatible with the average

Caricatures

I created caricatures by extrapolating my face away from the average cordinates.

Bells and Whistles: Caricatures in PCA space

It turns out that more than 92% of the variance is explained by just the first two principle components. Interestingly, when we plot the components themselves, it starts to become clear that they actually represent facial orientation.

It is natural to wonder if caricatures in this space would produce rotations. As expected, they do:

This is extremely interesting from a manifold learning point of view. Maybe there is a technique that preserves the facial structure even better than PCA. However, in the interest of time, I did not pursue any more advanced manifold learning techniques.