Face Morphing

CS194-26 Image Manipulation and Computational Photography Project 4.

A CS 194-26 project by Kristin Ho, cs194-26-aai

In this assignment I produce a "morph" animation of my face into someone else's face, compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.

A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.

A triangulation is then defined on the correspondence, and pixels in each triangle are warped to the desired midway position using an affine transformation that I compute as a change of basis matrix based on the triangle vertices.

Running the Code

These images were created with several Python 3 scripts written from scratch. Matplotlib, numpy, scipy, and skimage are needed to run the code. The Danes data set was used to generate an example of an average face.

Midway Face

The main task in warping the faces into the average shape is implementing an affine warp for each triangle in the triangulation from the original images into this new shape. This involves computing an affine transformation matrix A between two triangles. Once the two images are warped into the same shape (one triangle at a time), their values are cross-dissolved.

Kristin (me).
Kaudrey or Aristin.
Audrey Hepburn.

Of the midway images I computed, a cross between my friend and her younger sister is one of the most convincing.

Jane (the older sister).
Sane or Jally.
Sally (the younger sister).

I also thought the androgynous looking George-Audrey cross wasn't too bad.

George Clooney.
Gaudrey or Aeorge.
Audrey Hepburn.

Morph Sequence

I produce a warp between im1 and im2 using point correspondences the user provides through clicks and a triangulation structure generated from their midway shape. The warp fraction and dissolve fraction are incremented through 45 frames, and control shape warping and cross-dissolve, respectively.

Kristin (me).
Kaudrey or Aristin.
Audrey Hepburn.

The "Mean" Face of a Population

I use the annotated Danes data set to compute the mean face of the Danish population. I first found the average face shape, warped each image to the average shape, and then cross-dissolved all images.

The "average" Dane.

Here are examples of samples from the data set, morphed into the average shape.

Random female dane.
Female dane warped to Average Dane face shape.
Average Dane.
Random male dane.
Male dane warped to Average Dane face shape.
Average Dane.

My face unflatteringly warped into the average Danish shape (changed the point correspondences slightly to account for differences in image).

Kristin (me).
My face warped to Average Dane geometry.
Average Dane.

The average dane warped into my face shape.

Average Dane.
Average dane warped to my geometry.
Kristin (me).

Caricatures: Extrapolating from the Mean

A "caricature" of a face can be derived by extrapolating from the population mean. Here we extrapolate from the Danish mean calculated in the last step.

Kristin (me).
Kristin caricatured to be less like Mean Danish face (emphasizing the differences).
Average Dane.

Bells and Whistles

As a fun addition to face morphing, I decided to try morphing the faces/bodies of two cartoon characters with very different features. Jake and Lady Ranicorn from Adventure Time!!! I've shown both the source images, midway image, and the morph.