CS194-26 Project 4 - Face Morphing

Parsa Fereydouni - cs194-26-agy

Overview

In this assignment we will produce a "morph" animation of one face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature.
Morphing is done by: (1) Aligning, and (2) Cross-disolve.

(1) Aligning: To align we apply an affine transformation on the source traingle to adjust it's shape to the target traignle. Our affine transformation can be described as 3 basic transformation which stretch, rotate and translate. Our resulting transfomation, represented as a (change of basis) matrix is the following:

$$T_{affine} = \left[ {\begin{array}{cc} a & b & c\\ d & e & f\\ 0 & 0 & 1\\ \end{array} } \right] $$

(2) Cross-disolving: Here we simply add the image's color such that it sums to 1

Additionally, in order to create a smooth morphing transition, we will generate a sequence with progressing adjustment value by shifting the triangulation points to one side rather than the other. The adjustment value is effectively the importance of each image with respect to our transfomation.

Defining Correspondences

We define key points corresponding between both images for triangulation purposes

In [143]:
 
In [238]:
 

Computing the "Mid-way Face"

We generate our results by generating a Mid-way Face and morphing our images to it.

1) computing the average shape (a.k.a the average of each keypoint location in the two faces)
2) warping both faces into that shape

Add midway points to each image

In [297]:
 

warp faces into the midway shape

In [298]:
 

3) averaging the colors together

In [299]:
 

The Morph Sequence

make a 45 frame transition from image1 to image2. Since each of the original images is one frame, we have 43 tansition frames that are created by repeating the above process on different weighted averages of the two faces.

shown below is 9 of these images (every 5th image of the 45)

In [244]:
 
image #0
image #1
image #2
image #3
image #4
image #5
image #6
image #7
image #8

Here's a gif of all 45 frames:

The "Mean face" of a population

In this part I used the Danes public database to generate an average face. The subset I used for the mean face is all neutral, male images.

To get the Mean Face, I repeated the same procedure as before for getting the Mid-way Face. The difference here is that instead of taking the average of two faces, we now have 33 faces

In [300]:
 

Caricatures: Extrapolating from the mean

Producing a caricature of my face by extrapolating from the population mean calculated in the last step

In [305]:
 

The following caricatures are achieved by alpha of -0.5 and -0.75 respectively:

Apparently I have big eyebrows! Which means when I exagerate my facial features with a negative alpha in this process, I end up with huge caterpillars over my eyes...

Bells and Whistles: change smile of your face

For this part, I take the average of 33 smiling Danes from the public database. Then I give myself an average Danish smile :)

Here you can see the mean smiling face:

In [306]:
 

I selected the correspondence points to focus mainly on the mouth area. The smiling face shape that I used was a combination of 25 percent of my face's shape and 75 percent of the Mean Smiling Faces's shape. The following is the result: