CS-194 Project 3

By: Calvin Chen

For this project, I worked on morphing faces, particularly mines to George Clooney's, computed the mean face for a variety of Danish people, and extrapolated from that population mean to create caricatures of myself.

Defining Correspondence

For this section, I collected points for my face and George Clooney's in order to find a correspondence between facial features, and constructed a Delaunay Triangulation on the average face.

Faces, Points, and Triangulations

George Clooney Triangulation and Points

My Triangulation and Points

Computing the "Mid-Way Face"

For this section, I broke down the problem into the following algorithm.

Algorithm:

  1. Find the midway shape between the two averages by taking the average of the two sets of feature points and compute the Delaunay Triangulation on that.
  2. Take the simplices, or indices of triangles, from the result and extract the estimated coordinates from the original images.
  3. Create an interpolation for each of the original images over all 3 colors channels (RGB). This way, when translating points from the mid-way face back to the original ones, we're able to better estimate RGB values even if the translations don't end up at exact pixel coordinates.
  4. For each of the triangles, calculate the affine transformation between the midway face and the original face.
  5. For each of the triangles, map the pixels within each triangle back to the original images using the affine transformation matrix for that specific triangle, and use the interpolation functions returned for RGB to determine what pixel value should exist in the mid-way shape for that location.
  6. Take an average of the two values from both images for each color channel.
  7. Output the mid-way shape with its averaged colors.

George Clooney's Face

Midway Image

My Face

The Morph Sequence

For this section, I constructed different GIFs to morph form one face to another. The algorithm and steps I used are listed below.

Algorithm:

  1. Calculate the midway/intermediate shape using a warp_frac parameter that varies from 0 to 1 that weights results in the following weight: A warp_frac + B (1 - warp_frac).
  2. Warps this weighted, mid-way shape from the original image.
  3. Calculate the pixel values in the warped face with a dissolve_frac parameter that varies from 0 to 1, with similar weighting of A dissolve_frac + B (1 - dissolve_frac) for each RGB color channel.

Morping from Paul Rudd (Ant-Man)'s face to my face

The "Mean Face" of a Population

For this section, I calculate the "mean face" of a variety of labeled images of Danish people. The algorithm and steps I took are listed below.

Algorithm:

  1. Compute the average face shape of the whole population (n=40)
  2. Morph each face into this average shape using a slightly modified algorithm from above.
  3. Compute the average face of the population and display.

Average Face

Different examples of people's face morphed into the average face

Original

Warped

Original

Warped

Original

Warped

My face warped into the average geometry

The average face warped into my geometry

Caricatures: Extrapolating From the Mean

For this section, I calculated caricatures by extrapolating from the mean face of the Danish face dataset.

Algorithm

  1. Weight the average points by alpha * (my_face_pts - avg_pts).
  2. Morph this new, weighted face into the average face with different alpha values.

Caricature with Alpha = -1

Caricature with Alpha = 1.8

Bells and Whistles

For this section, I made a YouTube video with some classmates morphing our face over a music video.