CS194-26: Face Morphing

Overview

The goal of the project is to produce a morph animation between different faces, computing the mean of a population faces and extrapolate from the population mean to create a caricature of myself. A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. By defining a correspondence between the two pictures, we can map from eyes to eyes, mouth to mouth, chin to chin, ears to ears. To get the smoothest transformation, we need to use affine transformation to warp the two pictures to their average shape and then use cross-dissolve of the image colors to get the smoothest transformation.

Part 1: Defining Correspondence

First, I need to define pairs of corresponding points on the two images by hand. In the example below, I created the corresponding points by manually selecting the head shape, the face shape, eyes, eyebrow, nose, mouth, chin and neck with 44 points. After that, I created the Delaunay triangulation with those points for each image and also the triangulation at midway shape to lessen potential triangle deformations.

original image 1

original image 2

Delaunay triangulation

Triangulation at midway shape

Part 2: Computing the "Mid-way Face"

Before computing the morph sequence, I computed the mid-way face of the two images. First, I compute the average shape of the image, which is calculated by taking the average of each key point location in the two faces. After that, I warped both faces into that shape and average the color of the two images together to get the mid-way face. During the warping process, I computed the affine transformation matrices for each triangle and applied the transformation to transform each pixel of the original image to the average shape.

The mid-way average face

Part 3: The Morph Sequence

To produce the morph sequence, I use two parameters warp_frac and dissolve_frac to control shape warping and cross-dissolve, respectively. During the 50 frames, both warp_frac and dissolve_frac increased from 0 to 1 with an interval of 0.02. In this way, instead of computing the mid-way face, we can gradually see the transformation form one image to the other.

Morph Sequence

Another Example: Input

Another Example: Morph Sequence

Part 4: The "Mean face" of a population

4.1 The mean face of population

The mean face of population is calculated by first warping all images to the average shape and then finding the average of all of the warped face. In the example below, I chose the Danes dataset and compute the average of all the males facing to the front in the dataset. We can also see example of three different man being warped to the average face shape.

The average face of men	Man 4	Man 6	Man 13
The average face of men	Man warped to average shape	Man warped to average shape	Man warped to average shape

4.2 My face and the mean face

I also tried to warp my face into the average geometry and warp the average face into my geometry. In the examples below, I warped my face into the average geometry by defining the corresponding points and apply affine transformation to each of the triangles in order to warp my face to the mean face. The picture looks a little weird due to different expression of the two images but we can clearly see from the eyes, the chin and the face shape to see the morphing.

To create a caricature of my face, I first find the caricature shape by interpolation. I subtracted the average points from my own image points and then add the difference back to my own image points in order to interpolate and create the caricature shape. After that, I computed the affine matrices from my image points to the caricatured points and warp my image to the shape in order to create the caricature of myself. We can see that since in the original image of my face, my eyes and nose are larger than the mean face, those features become more significant in the caricatured face.

Part 5: Bells and Whistles: Changing Gender of Image

Example 1

For this Bells and Whistles I worked on changing the gender of my friend’s faces. In the first example, which is the example in the morphing sequence above, I first showed the average of two images by just morphing the appearance. We can see that when we are only morphing the appearance/color, the result is a failure since the images aren’t aligned at all. Then, I showed the result of morphing just the shape. We can clearly see the male features when my friend’s face is morphed to her favorite actor’s face and also some female features when the actor’s face is morphed to my friend’s face shape. For averaging both appearance and shape, the result is shown in the mid-way face in part 2 above.

Example 2

In the second example, I created a morphing sequence between one of my other friends and her favorite actor. When morphing just the appearance, we get the similar problem of misalignment. And when morphing just the shape, we can see the new features in the morphed pictures in the eyes, mouth and chins. Finally, by morphing both appearance and shape, we get the mid-way average shape of the two images.

Face Morphing

CS194-26 Image Manipulation and Computational Photography

Jingxi Huang cs194-26-aap