Face Morphing

by Huy Hoang

Overview

The goal of this project is to create a morphing animation between two different faces, compute and display the mean of a population of faces, and extrapolate from a population mean to create caricatures of images.

Morphing - Problem Statement and Intuition

Input: two images A and B containing human faces

Output: a fluid GIF showing how image A morphs into image B

Morphing comprises of the following steps:

1. Defining Correspondences: this step ensures that the features (e.g. eyes, mouth, nose, etc.) of the two images can be aligned and eliminates any ghosting effects
2. For each frame at time t in the range [0,1]
   • Compute the intermediate shape by using linear interpolation on each feature pair
     
     
   • Cross dissolve the color by interpolating the two images

Defining Correspondences

I manually defined 76 pairs of correspondences in both images, including 8 points in the corners and edges of the images which will help give a smoother transition. After getting these points, I then defined a triangular mesh over the points in each image using Delaunay triangulation. Triangular mesh gives us triangle-to-triangle correspondences between the two images, including all pixels contained in the triangles.

Computing the Mid-way Face

Before moving on to the whole morphing sequence, I have to compute the Mid-way face of the two images, which is achieved by warping the images to their average shape and interpolating the colors. Specifically, I use the inverse of the transformation matrix T that goes from source image to the midway image in order to find the corresponding points in the source image and perform bilinear interpolation on the coordinates to get the closest pixel value.

This approach, also known as Inverse Warping, eliminates cases where holes (black squares in the photo below) may show up in the warped image since pixel values at locations that no coincide with pixel coordinates can be obtained from the source image using bilinear interpolation.

Below is my result on the photos above

Morphing Sequence

The fun starts now! Armed with everything I've implemented in the previous steps, I can move on to create a morphing sequence for Brad Pitt and Edward Norton in Fight Club.

At each timestep t in the range [0,1]:

• I get warp_frac and dissolve_frac, which controls the degree of warping and cross dissolving between the two images, respectively
• warp_frac determines the intermediate shape that the images are warped into
• dissolve_frac determines how much of each image A and B is cross dissolved into the warped image
• At time t = 0, we'll get image A and at time t = 1, we'll get image B

Here's another fun morphing sequence for the Breaking Bad fans out there!

The "Mean Face" of a Population

Here, I'm computing the average face of the Danes dataset. In addition, I'm also warping some of the faces in the dataset into the average face.

Here are some fun results when warping an image outside of the dataset into the average face.

Caricatures: Extrapolating from the Mean

Using the average face I got from the Danes dataset, I can generate caricatures by adding the weighted difference betweeen my source image and the average face and adding it back to my source image. Here, I'm weighted the difference with parameter alpha, where alpha can be less than 0 or larger than 1.