CS 194-26 Proj3

Part 1: Defining Correspondences

To morph two images, we need to do two parts: warping and cross-dissolving. For warping, we need the geometry of two images with a one-to-one relationship, as well as an average shape. To get the one-to-one shape, I manually define a set of labels in two images with the same order. For each of the faces, I include 50 label points, focusing on eyebrows, eyes, noses, mouths, and jawlines. Given these label points, we then use the Delaunay function to generate triangulations. To make sure the triangulations work well on both images, I generate triangles on the average shape. Then, I apply them back to individual label points of two images.

Below are labeled points and corresponding triangles of my photo and the given George photo, along with original photos.

Original photos

Photos with points and triangulations

Part 2: Computing the "Mid-way Face"

To implement the actual morphing sequence, it is necessary to include two main parts: warping, and cross-dissolving. Cross-dissolving is just an interpolation of two images' color, while the warping function is more complicated.

To eliminate holes caused by missing pixels, I implement the inverse warping function. This inverse warping function takes the following parameters: input image, its original shape arrays, one target shape array, and the corresponding triangles. It iterates over each triangulation and computes the triangles in each shape array, and calculates the affine transformation matrix between every two triangles. Using the transformation matrix, I record inside pixels and corresponding pixels in the original image. After saving all pixels, I use RectBivariateSpline to interpolate pixels in between different integers and generate the warped image.

Given this warping function, the midway image is not hard to compute: given two images and two shape arrays, I warp each image toward the average shape using warping factor 0.5, and cross-dissolve the two images using dissolve factor 0.5.

Below is the corresponding midway image, along with two original images.

Han vs. Midway vs. George

Part 3: The Morph Sequence

To get the whole sequence, I follow a similar process, but inside a loop of the number of frames. For each frame, I calculate a specific warping factor and dissolving factor. For two original images, I calculate a shifted average shape based on the warping factor, warp both images toward the weighted average shape, and cross-dissolve two warped images with the dissolving factor. I repeat this process over each frame and save all recorded frames into a gif file. Note that because the gif file is too large, I upload it to youtube after changing the file format.

For the following video, I include 45 frames from the first original image to the second one and another 45 frames from the second to the first. Each frame is saved with a duration of 1 over 30 seconds.

Part 4: The Morph Sequence

The way to calculate the mean face of a population is very similar to the way to calculate a mid-way face. Instead of warping two images into the average shape, the mean face of a population requires each image to be warped into the average shape, and again cross-dissolve every warped image.

Below is a mean face of the whole Danes dataset. For clarity, I only include pure passport-style images, which are named "1m" or "1f".

Mean Face of Danes Population

Here I also include two examples from the Danes dataset and the results of warping these faces into the geometry of the mean face.

Origibal Examples

Warped Examples

Also, I want to show two more examples: my face warped into mean face geometry, and mean face warped into my face's geometry. Note here I reselect 50 label points on both the photo of my face and the mean face image in the same order on the same features. However, this still leads kind of unrealistic faces(kind of scary actually).

I warped into mean vs. Mean warped into me

Part 5: Caricatures: Extrapolating from the mean

To extrapolate from the mean face, I first calculate the vector from my face geometry into the mean face geometry as described in the lecture. Then, to extrapolate I use my face + alpha * vector and record examples with different alpha values.

alpha = 0.5 vs. alpha = 1.2

alpha = -0.2 vs. alpha = -0.7

Bells and Whistles

Change Gender and Emotion

Here I choose to change the gender and emotion of my face, by calculating the average shape of my image and some an image of the female and an image of angry people. These images are acquired from http://www.faceresearch.org. For each of these tasks, I include four examples: original image, "matching shape only", "matching apperance only", and both "shape and apperance matching".

Here are results of changing gender.