CS 194-26 Project 3: Face Morphing

Reini Lin

Overview

The purpose of the project was to create a smooth morph between different images. In the first part, I sought to morph from one face into the other by labelling correspondences, creating triangles from the correspondences, and applying an affine transformation to morph one triangle into another. In the second part, I computed the mean face of a Brazilian face dataset and used it to create a caricature of my own picture.

Correspondences

I started by plotting correspondences on both faces. To morph my face to George Clooney’s, I used 22 correspondence points (L = left, C = center, R = right):

For my face morph with Deepshika, I used Efros’ recommended 43 points to keep things uniform with the morphs created by the other students.

Above: George with his original correspondences plotted, George with the mean points plotted

Above: Me with my original correspondences plotted, me with the mean points plotted

Midway Face

I followed the following algorithm to compute the mean face:

  1. Compute the mean pointset by averaging my correspondences with George’s. For midway face, alpha = 0.5, where alpha is the cross-dissolve factor and is a value in [0, 1] denoting how much to weight image2 in the final output.
  2. Morph both faces into that pointset by iterating through each triangle in the triangulation found by the Delaunay algorithm.
  3. For each triangle, find T1, the transformation matrix from source image1 triangle coordinates to destination triangle coordinates. Apply the inverse warp T1(^-1) to go from all coordinates within the output triangle back to the original points in image 1. Use the original coordinates from image1 to move the values at the original coordinates into the coordinates of the destination triangle. Repeat for image2.
  4. Once all triangles have been created and put together into a resulting image, average the resulting images of both faces. For the midway face, the weighted average is just an even weighting, with alpha = 0.5

The image below shows the output of the halfway point, computed on the morph from my face to George’s. Some areas, like the glasses/hair/teeth, can’t be resolved very well because of the nature of the original images. However, alignment of key features such as eyes, mouth, nose, ears, and overall face is pretty good.

Morphing

I created the morph by applying the algorithm described in midway face, but using a gradually increasing value of alpha for each frame. As per the spec, I created 44 intermediate images between the image of Deepshika and the image of me. Altogether, there are 46 images, with the first image identical to the original image of Deepshika and the last image identical to the original image of me.

Mean Face of a Population

The mean face was created using the Brazilian dataset found here. I used the first set of normalized forward-facing images with smiling as well as the provided annotations.

To compute the mean face, the following steps were taken:

  1. Compute the mean pointset by averaging all the pointsets for all faces.
  2. Morph all faces into the mean pointset. Images provided below.
  3. Take an average of all results from step 2.

Above: Person 1 before warping and after warping into the mean pointset.

Above: Person 2 before warping and after warping into the mean pointset.

By taking an average of all the faces warped into the mean pointset, I created the mean face of the population.

I also morphed my face into the mean pointset and morphed the average Brazilian face into my pointset.

Caricatures

The caricature was created by warping my face away from the average geometry of the population. I used a value of alpha = -0.6 to caricaturize my image of myself.

Bells & Whistles

I morphed my face from Deepshika’s, and the videos created by other students in the class were strung together into one video, below. (Link is also here in case the embedding doesn’t work: video).