Face Morphing

by Pauline Hidalgo

Overview

This project involves image morphing and warping! In the first part of this project, I use shape-warping and cross-dissolving to morph between myself and Michael Cera. In the second part, I use faces from the FEI face database to warp images and create caricatures using the average face.

Defining Correspondences

To begin morphing, I define 44 points on mine and Michael's face. These points act as correspondences between the two faces during morphing, so it is important to define them in the same order. To prevent the backgrounds getting clipped out during the morph, I also add 4 correspondence points in the corners of each image.

Computing the "Mid-way" Face

Warping between two face shapes involves computing affine transformations between triangles. To ensure the triangles don't become too skinny throughout the morph, I base the Delaunay triangulation on the average shape between myself and Michael Cera (computed by taking the average of the correspondence points). Computing the mid-way face involved inverse warping the average to the original faces to get the corresponding locations, then cross-dissolving (also averaging) the colors from both images. I used bilinear interpolation to get the pixel brightness for pixels landing "between" multiple corresponding pixels.

triangulation from average pts

original

original

mid-way face


The Morph Sequence

Then, I computed a morph sequence by applying the same procedure as above, except using a weighted average based on the frame number for the shape warp and cross dissolve as opposed to a normal average. Weights are: dissolve_frac = warp_frac = frame_no / 45 for frame_no's 0 to 45.


The "Mean face" of a population

Here, I computed the average face of 200 smiling faces in the FEI face database. First, I averaged all of the correspondence points, used those to create a triangulation, then warped the faces from the dataset into the average shape. Once each face was warped to the average shape, I averaged the pixel brightnesses of all of the images to get the overall average face. I also used my own image to warp my face to the average shape and the average shape to my face. To ensure my photo was consistent with the FEI faces, I recropped my photo to the correct dimensions, used grayscale, and defined new correspondence points consistent with the other .pts files.

average face

original

my face warped to the average shape

the average face warped to my shape

The way the faces above get blown up and shrunk indicates that I probably should've cropped my image a little closer to my face. However, you can see that my smile gets bigger when warping my face to the average smiling face shape! Similar (and more subtle) results can be seen using faces from the database:

original

warped to average

original

warped to average


Caricatures

To create a caricature of my face, I get the deviation of my face shape from the average shape computed in the last part. I add alpha * this deviation back to the average face shape, then warp my face to these extrapolated correspondence points to get a caricature. A similar result is achieved by setting warp_frac to be negative when warping my face to the average shape. As alpha increases, my chin becomes pointier, my nose becomes smaller, and my face generally moves lower and lower. Because I'm not smiling as wide as the average face, I start to frown too.

alpha = 1 (original)

alpha = 1.4

alpha = 1.7

alpha = 2


Bells & Whistles

For bells and whistles, I changed my ethnicity by morphing my face with the average french woman. Morphing both shape and appearance definitely gives the most convincing result, and morphing only the shape or color gives more caricature-like results. Average-French me has a pointier nose, higher cheekbones, thinner eyebrows, and an overall smoother look due to the averaging.

average french woman's face

correspondences

my face (same img from previous section)

correspondences

morphing both shape and appearance

morphing just the shape

morphing just the appearance


Finally, I made a morphing music video with characters and music from Scott Pilgrim vs. The World, which is a pretty fun movie!