Overview

In this project, I did manual selection of correspondences between images and used Delaunay triangulation, affine transformations, and cross-dissolving to warp faces to shapes of other faces as well as create morph animations between faces. I also computed the mean face of a population and used it to extrapolate and create caricatures.

Part 1: Defining Correspondences

First, I cropped and rescaled an image of myself and an image of my friend Angela to have the same dimensions, and so that our faces have similar size and aspect ratio. I then used the provided online tool to select 61 correspondence points for each of the two images, and added the four corners of the image (so that every pixel is located in a triangle) to ultimately obtain a total of 65 correspondence points for each image. Finally, I generated a Delaunay triangulation at the mean of the two sets of correspondence points.

~ results ~

Chloe
keypoints & triangulation

Angela
keypoints & triangulation

Part 2: Computing the "Mid-way Face"

To compute the mid-way face for me and Angela, I did the following steps:

  1. Compute the average shape
    avg_shape =  1⁄2 (im1_pts + im2_pts)
  2. Calculate the inverse of the affine transformation matrix A between the original triangle in each image and the corresponding triangle in the average shape. Do inverse warping with this matrix, using polygon to mask all the pixels.
  3. Cross-dissolve by averaging the warped images.

~ results ~

Chloe

Chloe-Angela midway

Angela

Part 3: The Morph Sequence

Using Part 2 and a range of weights in [0, 1] I created a morph sequence 56 frames, with duration of 30 ms per frame.


Part 4: The "Mean Face" of a population

For this part, I computed the average face for multiple images. I used the Danes dataset, which consists of 37 images of Danish individuals (30 male, 7 female).

This involved first parsing the dataset '.asf' files to extract the correspondence keypoints for each image, average all points to obtain the average shape, and warping each face in the dataset to the average shape.

~ some examples ~

below are 4 examples consisting of a pair of images
the left is an image of a Danish individual and the right is the individual's face warped to the average Dane face shape

example 1

warped

example 2

warped

example 3

warped

example 4

warped


I then averaged each of the warped images to obtain the average face for all of the images in the dataset (left), only the females (middle), and only the males (right).

average Dane

average female Dane

average male Dane


Using the previous results, I warped my face into the average face's geometry and the average face to my face's geometry. Initially, I only did this with the average face of all images in the datase (average Dane). The result was a little disturbing, so I also tried it with the average female Dane face. However, that produced equally disturbing results.

~ results ~

me

average Dane

me to average Dane

average Dane to me

me to average female Dane

average female Dane to me

Part 5: Caricatures - Extrapolating from the mean

I created caricatures of my face by extrapolating from the population mean in the previous part. I achieved this by choosing alphas outside the normal range of [0, 1], to exaggerate either my face's features (alpha < 0) or the average face's features (alpha > 1).

~ results ~

α = -0.5

α = 1.5

Bells and Whistles

> Gender Change

For my first bells and whistles, I changed the gender of my friend's face using the average Chinese male face I found on the web. I first rescaled and resized the two images and chose correspondence points between them. Then, I played around with three different methods of morphing my friend's face and the average Chinese male face face:

  1. Only morphing the shape (warp) - this involved warping my friend's face to the geometry of the average Chinese male's face.
  2. Only morphing the appearance (cross-dissolve) - this involved generating the average face warped to my face shape and then cross-dissolving.
  3. Morphing both shape and apperance (warp and cross-dissolve).

Angela

average Chinese male

~ results ~

shape only

appearance only

shape & appearance


> Music Video

I also created a morphing music video of my face at different ages (at less than 1, 2, 6, 10, 13, 15, 18, 19, and 20 yrs old). I accomplished this by stacking multiple morph animations (50 frames of duration 30 ms for each gif) together and converting the result to a mp4 video.