CS194-26: Image Manipulation and Computational Photography Fall 2018
Maddie Pahati, cs194-26-abz

Project 4: Face Morphing


Overview

In this project, we were given multiple tasks related to morphing and averaging faces. The goal was to morph one face into another, compute the average faces given a population of people, and then extrapolate from that mean to create a caricature of myself. For the morph, I used the senior pictures of myself and my younger brother, Jerrick.

Defining Correspondencies

To get the smoothest transformation, we defined correspondencies between one face and another that maps mouth to mouth, eyes to eyes, nose to nose, etc, keeping the ordering of coordinates consistent between the two faces. In addition, the four corners were added to the list of points.

Next we compute the Delaunay triangulation at the midway shape, which is the average set of points between face1's correspondencies and face2's correspondencies. The Delaunay triangulation maximizes the smallest angles and was computed using scipy.spatial.Delaunay. We use the midway shape to ensure the triangulation will be the same for both faces because it is possible to get different triangulations when running the Delaunay on two separate images.

Tri Midway

Computing the "Mid-way Face"

Once we have gotten all the keypoints and triangulation, the next step was to compute a "mid-way face" before computing the entire morph. Using the triangulation found in the previous part, I warped each face to that shape, and then averaged out the colors.

To warp each face to the average shape, I computed the affine transformation between each source triangle and its corresponding midway triangle. The affine transformation matrix was calculated using the following formula with inspiration from this Stack Overflow post.

Given transformation matrix T, matrix A with the target triangle coords, and matrix X with the source triangle coords,

Eqn

Where (a1,a2), (b1,b2), (c1,c2) are points of the triangle in the target image (the average shape) and (x1,x2), (y1,y2), (z1,z2) are points of the triangle in the source image, we can compute the transformation matrix T by multiplying X by A inverse.

New Eqn

The corresponding pixel location in the source image can then be computed by multiplying the target point by the affine transformation matrix we just calculated.

I used draw.polygon to find all the target points in each triangle, applied the transform to the corresponding point in both source images, interpolated the value at the point, averaged the two values together and then copied the average in the target image.

Midway

The Morph Sequence

Here we produce a warp that transforms my face smoothly into my younger brother's face. I produced 45 frames where each frame is a morph of the two faces but applied with a different weight from [0,1].

Morph

As you can see, the hair and the chest area are not as smooth which is due to the low number of correspondencies in those areas. This can be fixed by adding more keypoints.


The "Mean Face" of a Population

Using the Danes dataset, I computed the average face shape for the men and the women, morphed each face to their respective average shape, and the averaged all the images to get the "average" man and the "average" woman.

Avg Man Avg Woman

I then morphed my face to the geometry of the average smiling woman and vice versa.

Me to Mean Mean to Me

Caricatures

A caricature is an image where certain features are exaggerated. I created a caricature of myself by extrapolating from the average smiling woman.

Caricatures

My caricatures, especially when alpha=1.5, looks quite qnarly but it makes sense because my shape is rather different than the average Danish woman.

Bells and Whistles

Make a Morph Based on a Theme

I dug up all my old school pictures and made a morph of me throughout the years from kindergarten to my high school graduation.