CS194 Project 3: Face Morphing

Cesar Plascencia Zuniga

Overview

Throughout my life I have dreamed of being a professional futbol player and today I can finally live out the dream with the power of face morphing! Before the morphing begins, we must define correspondence points in both images so that facial sections line up with each other. Without correspondence points, there would not be an actual 'warp'.

After computing the correspondence points, we will connect them with triangles using a Delaunay triangulation.

Defining Correspondences

Me
Messi
Alignment Points (w/ Delauney)
Alignment Points (w/ Delauney)

Computing the "Mid-Way Face"

Before we start to morph the images, we will first find the mid-way face. This mid-way represents the average of both images. We begin by taking the average of our correspondence points and then creating the Delaunay triangulation over these average points.

Once we have the triangulation, we compute affine transformations for each triangle in the image. The idea here is that we will use the affine matrix to warp triangles in the source image to triangles in the mid-way image. After warping the shape our source image, we need to extract the actual pixel/color values for our mid-way face. I used RectBivariateSpline to compute an interpolation function based off the source image. With this function I took pixel-positions in the mid-way image, applied an inverse affine transformation to find the corresponding pixel in the source image, and finally used the interpolation function to extract the pixel value of this inverse mapped pixel.

The above process is repeated for every single triangle in order to obtain our mid-way face.

Delauney of Mean Shape
My Midway Morphed Face
Messi's Midway Morphed Face
Both Midway Faces with Cross-Dissolve

The Morph Sequence

The mid-way image involved using 50% of our first image's shape/color and 50% of our second image's shape/color. To create a smooth morph sequence, all we have to do is change these percentages for different frames. For example, a 5-frame image would consist of frames with 0% first image shape/color & 100% second image shape/color followed by 25% & 75%, 50% & 50%, 75% & 25%, and lastly 100% & 0%. Below is a result of a 30-frame image.

Cesar to Messi
My Transition to Soccer Stardom

The "Mean Face" of a Population

Now imagine we had a variety of different faces to work with. It is possible to compute an average face over them. We will be using a dataset of Danish faces in order to compute the average face shape. Before finding the average though, we must warp all of the faces to the mean face. Dataset can be found here: http://www2.imm.dtu.dk/~aam/datasets/datasets.html

Average Dane Face Shape
Triangulation
Warped Image 0
Warped Image 6
Warped Image 18
Warped Image 36
Warped Image 19
Warped Image 31
Average Face

We can also take normal images and shape them to look like the mean!

Normal Cesar
Dane-Shaped Cesar
Cesar-Shaped Dane

Caricatures: Extrapolating From the Mean

So far, we have been working with data that already exists in the image. All of the 'alpha' values we have been using are between 0 and 1. The idea is that we will calculate the difference in the mean face shape and our face shape, then add this difference back to our original image. The amount that it is added back is proportional to alpha. Below are some examples.

Normal Cesar
Positive Extrapolation a = 1.25
Negative Extrapolation a = -1.25

Bells and Whistles

With the help of wonderful classmates we were able to create a class morphing music video! Check it out here -- CLASS MUSIC VIDEO