CS194-26 Project 4 - Face Morphing

Eilam Levitov - cs194-26-acx
This notebook runs on python 2.7

Overview

In this assignment we play with the affine transformation, morphing faces and making caricatures.
Morphing is done by: (1) Aligning, and (2) Cross-disolve.

(1) To align we apply an affine transformation on the source traingle to adjust it's shape to the target traignle. Our affine transformation can be described as 3 basic transformation which stretch, rotate and translate. Our resultant transfomation, represented as a (change of basis) matrix is as such:

$$T_{affine} = \left[ {\begin{array}{cc} a & b & c\\ d & e & f\\ 0 & 0 & 1\\ \end{array} } \right] $$

(2) Here we simply add the image's color such that it sums to 1

Additionally, in order to create a smooth morphing transition, we will generate a sequence with progressing adjustment value by shifting the triangulation points to one side rather than the other. The adjustment value is effectively the importance of each image with respect to our transfomation.

Finally, we will further explore the face morphing domain, averaging groups of people and pin-pointing facial features, specifically smiling :)

Follow the notebook for more specific details on the process of this project!

Defining Correspondences

We define key points corresponding between both images for triangulation purposes

Procedure

In [26]:
# 1. Load images
In [27]:
# 2. Load (manually) pre-selected points

Result

In [29]:
# Display

Computing the "Mid-way Face"

Rather than morphing from one end of the spectrum to the other, we improve our results by generating a Mid-way Face and morphing our images to it.

Procedure

In [30]:
# 1. Calculate Mid-way Face points
In [31]:
# 2. Triangulate Mid-way Face points using Delaunay Triangulation (maximize smallest angle)
In [32]:
# Display images with Mid-way Face points
In [33]:
# 3. Morph original images to Mid-way Face

CPU times: user 1.14 s, sys: 19.5 ms, total: 1.15 s
Wall time: 1.22 s
CPU times: user 1.05 s, sys: 17.4 ms, total: 1.07 s
Wall time: 1.11 s
In [39]:
# Display morphed images
In [35]:
# 4. Combine colors to generate Mid-way Face

Results

In [37]:
# Display

The Morph Sequence

Shifting the traingulatin points by incremently to make a nice smooth transition from face A to face B. This means we need to make a morph for each frame, thats 43 transformations!

Procedure

In [42]:
# 1. Morph images with progressing ratios
In [46]:
# Display (Selected) Progressions

Morph sequence #0, ratio =  0.976744186047  x Andrey +  0.0232558139535  x Eilam

Morph sequence #7, ratio =  0.813953488372  x Andrey +  0.186046511628  x Eilam

Morph sequence #14, ratio =  0.651162790698  x Andrey +  0.348837209302  x Eilam

Morph sequence #21, ratio =  0.488372093023  x Andrey +  0.511627906977  x Eilam

Morph sequence #28, ratio =  0.325581395349  x Andrey +  0.674418604651  x Eilam

Morph sequence #35, ratio =  0.162790697674  x Andrey +  0.837209302326  x Eilam

Morph sequence #42, ratio =  0.0  x Andrey +  1.0  x Eilam
In [53]:
# Saved images and generated .gif via photoshop (anaconda paths ftw...)

<matplotlib.figure.Figure at 0x103eb3bd0>

Result

Click me! (Giphy)

The "Mean face" of a population

In this part we used a public database to generate an average face. Luckily, all coresponding points were preselected for us.

Procedure

In [255]:
# 1. Load images; Display example
In [256]:
# 2. Load points and take average; Display example
In [257]:
# 3. Triangulate Mid-way Face points; Display example
In [258]:
# 3. Morph images to average, and combine colors

Results: The average brazilian, male, smiling, computer scientist

In [311]:
# Display

Caricatures: Extrapolating from the mean

Here we will use our smiling man right above this cell to make my picture more lively.

Procedure

In [328]:
# 1. Load images; Display
In [329]:
# 2. Load pre-selected points
In [330]:
# 3. Calculate Mid-way Face points
In [334]:
# Display
In [331]:
# 4. Triangulate Mid-way Face points using Delaunay Triangulation (maximize smallest angle)
In [333]:
# Display Mid-way Face points on each image
In [335]:
# 5. Morph original images to Mid-way Face

CPU times: user 1.08 s, sys: 17.4 ms, total: 1.1 s
Wall time: 1.14 s
CPU times: user 1.58 s, sys: 28.8 ms, total: 1.61 s
Wall time: 1.74 s
In [336]:
# Display

Results, with 3 different values

In [349]:
# Display 1
In [352]:
# Display 2
In [353]:
# Display 3

Random political remark because this is berkeley:

Click me!