Project 4: Face Morphing

Introduction

A face morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. Done over time, this creates a video of one face morphing into another. Doing a cross dissolve is easy, but warping the image shape is quite a bit more difficult. We accomplish this by creating sets of corresponding points in the two face images and triangulating the image based on those points. An affine transformation is computed for these triangles, and that allows us to morph between two images.

Defining Correspondences

Correspondence points were defined on the two source images. 45 points were chosen to be a good representation of the major features of a face, plus 4 (one for each corner). A Delaunay triangluation is computed on the average of the two sets of points. This is done because it is the closest point to both original source correspondence points that will minimize distortion.

Computing the "Mid-way" Face

Now that we have correspondence points and a triangulation, we can compute the affine transformation for each source image to the average triangulation. This is the first step to creating a morphed image.

The Morph Sequence

We can create a mid-way face at any point in the morph sequence by varying how much we blend between each source. We can use the equation:

out = (1 - t) * im1 + t * im2

While varying t from 0 to 1, to blend at any point in the image. A similar approach can be taken to blend between triangulations. If we save out every step of this process, we can create a video of the morph sequence.

The "Mean Face" of a Population

We chose a free to use dataset of faces of Danish people, located here. Correspondence points were already provided with these images! This makes it easy for us to calculate the average Danish face. We first find the average triangulation, and then morph a few selected Danes into the average triangulation to reimagine what these people might look like if their faces were shaped exactly like the average.

If we morph all of the original Danes into the mean triangulation, we can take an average of the images to compute what the average Danish person looks like (as represented by this sample of images).

We define a correspondence for my face, which allows us to morph my face into the geometry of the average Danish face. This allows us to imagine what I might look like if I were born as a Dane. We also morph the average Danish face into the geometry of my face, to see what an American-Dane might look like!

Caricatures: Extrapolating from the Mean

If we compute the difference between the mean geometry and my own geometry, then we have a delta that is a unique signature of my face. We can add this delta back to my own geometry to exaggerate my facial expressions and create a caricature of me. You can think of this as interpolating in the opposite direction of the mean.

I'm not going to lie, this disturbed me. I've lived my entire life not realizing that my nose is rather large and crooked, and all it took was CS 194-26 at UC Berkeley for me to figure that out.

Bells & Whistles: Changing Gender

After looking around on the Internet, we found this picture of the average human female.

We can define correspondences for it, allowing us to morph between my face and that of the average female. This should give us a rough idea of what I might look like with a more femenine face.

Reflections

Although the caricature of my face implies that I have a crooked nose, it is also possible that it could be that my head was not looking strait at the camera when the picture was taken. Any rotations of my face would create a delta that would move parts of my face away from the center, as the average face is very symmetric. I keep telling myself that this is it, and its not that I have a crooked nose.