CS194-26 Project 3 FACE MORPHING

You can fine a more detailed explanation of the project here.


In this assignment you will produce a "morph" animation of some one's face into someone else's face, and compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.

Defining Correspondence

To do so, for some pairs of images, we need to define the corresponding points by hand. I did this by using 'cpselect' package in python that basically does what cpselect does in matlab. Then, I computed Delauny triangulation using scipy.spatial.Delauny function, which results in triangulation like the following example.

An example of triangulation.

Mid-way Face

Now with the images' corresponding points and their triangulation, we are ready to proceed. With them, we can compute the 'mid-way face' of two images. We can do this by 1) computing the average shape (a.k.a the average of each keypoint location in the two faces), 2) warping both faces into that shape, and 3) averaging the colors together.

To 'warp' an image to another, we first compute the transformation matrix that maps the vertices of a triangle to those of the corresponding triangle. That is, given [x1, y1, 1] where x1, y1 are the coordinate of a vertice and [x2, y2, 1] where x2, y2 are the coordinate of the corresponding vertex, we need to compute matrix T such that T [x1, y1, 1]^T = [x2, y2, 1] for all 3 pairs of vertices. We can simply do this by construting two 3x3 matrices A and B, each with a triangle's 3 vertices as its columns, and compute T such that TA = B. We can do this by computing the inverse of matrix A. i.e., T = BA^-1. Since not every matrix is invertible, for stability compute the pseudo-inverse.

A picture of George Clooney
Mid-way Face
A picture of Brad Pitt
My friend Lucky
Fusion of Lucky and Jihan
My friend Jihan

Morph Sequence

Now, by producing multiple 'mid-way faces' by increasing the warp fraction and cross-dissolve fraction step by step (from 0 to 1), we can produce a sequence of morph.

The resulting gifs were pleasing to watch. Here are some fun results I had :)

Morph Sequence of George Clooney to Brad Pitt.
Morph Sequence of Lucky to Jihan.
Evolution of Eevee to Jolteon.

The "Mean face" of a population

By calculating the average shape of a population (or a sub population) and taking the average of the individual images warped to the average shape, we can get the 'mean face' of the population.

From FEI Face Database, I got 14 images for each of 200 brazilian individuals, a total of 2800 images.

Here are some individual smiling images warped into the average smiling shape.

#19 Smiling Brazilian
Warped #19 Smiling Brazilian
#21 Smiling Brazilian
Warped #21 Smiling Brazilian
#52 Smiling Brazilian
Warped #52 Smiling Brazilian

Now, by adding up all the warped images as above, we can get the mean faces. Here are the mean faces.

The Mean Face of the Whole Population
The Mean Face of Smiling Ones
The Mean Face of Neutral Faces

I can warp my face into the average shape of Brazilians to make myself look more Brazilian. Or I can also do it the other way around!

Original Me
Me warped to Brazilian Average
Brazilian Average warped to my shape
The Mean Face of Brazilians

Caricatures: Extrapolating from the mean

One can create a caricature, a face with exaggerated features, by extrapolating from the mean-face. The followings are caricatures of myself I got by extrapolating from the mean-face of all the brazilian faces. One can see that my eyes, nose, mouth are centered more as alpha increases.

Original Me
Me with alpha=0.1
Me with alpha=0.2
Me with alpha=0.3

Bells and Whistles

I made a morphing music video of Agumon from the animation 'Digimon'. Although the morphing sequences are not perfectly natural due to differenes in orientations and shapes, I had to make this video the moment I heard about image morphing. I just had to.