CS 194-26 - Project 4: Face Morphing

Eli Lipsitz (cs194-26-acw)

Note: Click on any image to enlarge it.

Correspondences

I defined a standardized set of correspondance points on two images: one of my face, and one of my roommate's face, using a simple ginput based tool.

Template (Prof. Efros)
My Points
Roommate Points

Face Morphing

I implemented a simple algorithm for doing face morphs between two images (A and B) with correspondence points:

  1. Lerp the correspondence points from each image together
  2. Compute a Delaunay triangulation of the lerped points
  3. For each triangle, compute the affine transformation matrix to transform from each triangle in the lerped pointset to the corresponding triangle in the original pointset.
  4. For each image, produce an output image: for each pixel, look up which triangle (in the lerp pointset) the pixel is in, do the affine transformation for that triangle, and pick that pixel from the original image.
  5. Lerp the two output images together to produce the final image.

Next, I blended my face with my roommate's face using this algorithm:

My Face
Roommate's Face
Mid-Way Face

And then I created a 60-frame morph sequence:

Mean Face

I took the Danish face dataset and computed the average face, average male face, and average female face.

The process is very simple: average all of the correspondence points together to compute the average shape, warp each face to the average shape, and do a simple average of all of the warped image.

Interestingly enough, each individual warped face looks terrible, but the average is extremely visually appealing.

Dane 11
Dane 33
Dane 12
Dane 11 Averaged
Dane 33 Averaged
Dane 12 Averaged
Average Male Dane
Average Dane
Average Female Dane

Finally, I took my face and warped it to the average Danish shape, and I took the average Danish face and warped it to my shape. Both results are disturbing.

My Face, Male Danish Shape
Male Danish Face, My Shape

Caricature

I used the average male Danish face computed in the previous part to create a caricature of myself, by taking my face and extrapolating the shape from the average Danish shape, past my own face shape.

My Face (t = 1.0)
Extra Eli (t = 1.5)
Extreme Eli (t = 2.0)

I created another morph sequence, just of my face, from the average Danish face (t = 0.0), to my face (t = 1.0), to my extrapolated face (t = 1.5):

Class Face Morph

I participated in a 23-person face morph: we each cropped our face to the same general cropping, defined correspondance points, and then ran our algorithm to generate 45 frames of morphing from the previous person in the sequence to us.

My face is at ~0:26 to ~0:28: Here's the video on YouTube