Project 3

Face Morphing by Amy Hung

CS 194-26: Image Manipulation and Computational Photography, Fall 2020


In this project, I will be "morphing" my face into another person's face, computing the mean of a population of faces, and creating a caricature of myself by extrapolating from a population mean.

Defining Correspondences

The first step in creating a morph is defining corresponding pairs of points between the two images. I first prepared the images by resizing them to be the same shape, and roughly aligned the faces relative to the overall image. I then used cpselect to pick 64 points to align on, concentrating points around crucial facial features (eyes, eyebrows, nose, mouth). I also added points in each corner to include the background in the morph. Using the chosen points as vertices, I then used scipy.spatial.Delaunay to create a triangulation pattern on the set of midpoints from the two images (to lessen potential triangle deformations), and applied the resulting triangulation pattern onto each image's points.
Original image of myself
Original image of George Clooney

Computing the "Mid-way" Face

After the triangulation pattern had been calculated for each image and the midpoint, we now warp each image to the "mid-way" face, or the triangulation of averaged points between the two images. I accomplished this by generating affine matrices for each pair of triangles (input image -> "mid-way" image), performing an inverse warp and interpolating the color for the warped image. With each image warped to the "mid-way" face shape, I then cross-dissolved the two images to get the resulting image:
"Mid-way" Face

The Morph Sequence

I then created a morph function to gradually morph and cross-dissolve the two images, and create a smoother transition. To control the degree to which each image is warped in each frame of the animation, I used two parameters: warp frac, which determines how much to warp each image towards the average face, and dissolve_frac, which determines how much of each image shows through. With this, I created 45 frames, where both parameters start at 0 and gradually increase to 1.

Note that in the morphing sequence, my long hair gradually appears in the cross-dissolve, rather than "growing" out of George's head, as it was not demarcated with corresponding points in the first step.
George to Amy Morph Sequence

The "Mean Face" of a Population

In this part of the project, I used a publicly available dataset of annotated Danish faces. Link to dataset here. I took the 37 forward-facing images, parsed in the annotated points for each image, and computed the average face shape fo the whole population. I then warped each of the faces in the dataset to the average shape, and cross-dissolved the warped images to display the average Danish face.
average Danish face
danish person #19
danish person #24
danish person #35
danish person #19 warped
danish person #24 warped
danish person #35 warped
My face warped into average Danish facial geometry
The average Danish face warped into my facial geometry

Caricatures: Extrapolating from the Mean

I then created a caricature of my own face, extrapolating from the Danish population mean. I did this by finding the difference between the average Danish face and my face, multiplying that difference by a factor alpha, and adding that back to the average image. Through the caricatures, you can see my defining facial features that differentiate me from Danish people amplified. In particular: my rounder head shape, smile, pointier chin, wider nose.
alpha = 1
alpha = 1.5
alpha = 2
alpha = 2.5
alpha = 3

Bells and Whistles: Parks and Rec Cast Music Video

For bells and whistles, I continuously morphed together the faces of Parks and Recreation's cast members. I selected the corresponding points for each image, and applied my morph functions for each pair of images, stitching them together for this final video.

Bells and Whistles: Classmates Music Video

I also participated in a continuous morph with some of my classmates!