CS194-26 Project #4
Face Morphing

OVERVIEW

In this project we morph one face into another, find the mean of a population of faces and create caricatures of our own faces using affine warping and some linear algebra.


DEFINING CORRESPONDENCES

The first step is to define a set of corresponding points on the two photos we want to morph into each other. I did this using cpselect. The key to choosing the right points is to select pairs of points that correspond to the same features on each image (e.g. jawline, nose, eyes, hairline). I had to go through a bit of trial and error here to see which areas needed more points to make the morph look more natural. For example, I realized the nose and eyes were very important because without well-selected points in those areas, the resulting midway morph would have slightly misaligned features.

COMPUTING THE "MID-WAY FACE"

This brings us to our next step, finding this midway morph. To find this, we first take the average of both images' correspondence points and use the average points to find a Delaunay triangulization. Then, we do an inverse warp on each image to warp them both into the average shape and average the colors together. With our two warped images, we can simply cross-dissolve them together (in this case, using weights of .5 and .5 since we want the mid-way face). I realized the importance of adding in the corner coordinates to my matrix -- without the corner points, the resulting image has some artifacts/borders that you can see.

No corners = bad!
Picking good correspondence points is also important...
(see the nostrils)

Perfect!

The idea behind this algorithm is to compute the affine transformation for every corresponding pair of triangles in the two images so that we can map every pixel from the source image to the target image. We use the provided mytsearch function to find which pixels fall within which triangles. We then use this information for interpolation.

THE MORPH SEQUENCE

Extending the concepts from the mid-way face, now we can completely morph one image into another by using various weights for image 1 and image 2. We do this repeatedly with evenly spaced warp_frac, dissolve_frac values for our frames in order to create a cool video morph!

THE MEAN FACE OF A POPULATION

Using the morphing function from above, we can generate the average face of an entire population! I used a subset of the Danes dataset. I used only the male faces in pose 1 (non-smiling) and discarded the images that had been corrupted to be only available in greyscale. I found the average shape of all of the faces and warped each individual face to match the geometry of this average face. Here are some examples of the individuals before and after morphing to the average geometry:



Here is the resulting average face!

Now, let's do some warping to see how my face would look if I were a Danish man, and vice versa.

Warping my facial geometry to a Danish male's:

Warping the average Danish face geometry to mine:

CARICATURES

Now that we have the population mean that we calculated in the last part, we can try to exaggerate those features by simply extrapolating. We do this by morphing with ratios that still sum to 1, but include a value > 1. For example, we can morph with the ratios -0.5 and 1.5 to exaggerate the appearance of Danish features in my face. Here are my results:

No warping:
Normal warping of my face into the
average Dane geometry:
Extrapolating with value 1.3:
Extrapolating with value 1.6:

BELLS & WHISTLES: DID YOU JUST ASSUME MY GENDER?

We can do some pretty cool stuff with appearance and geometry warping! I found an image of an 'average Korean male' online:



With this, we can change my gender! Here's what I would look like as a Korean man:

Morphing both shape and appearance:
Morphing just shape:
Morphing just appearance: