CS 194-26: Intro to Computer Vision and Computational Photography

Project3: Face Morphing

Yukai Luo



Overview

In this assignment I produced a "morph" animation of my face into my girlfriend Zoe's face, computed the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.

A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.

To start with, I took a pictures of yourself on a white background. My image is the same size and aspect ratio as Zoe's face. This will make your morphing result more pleasing to the eye.

I used my photo as Picture A and Zoe's photo as Picture B. I morphed still picture A into still picture B and produce 45 frames of animation numbered 0-45, where frame 0 is identical to picture A and frame 45 is identical to picture B. In the video, each frame is displayed for 1/30 of a second (ie. 30 fps). I created a video from my sequence of frames, as an animated gif.

Defining Correspondences

First, I defined pairs of corresponding points on the two images by hand. In order for the morph to work I have consistent labeling of the two faces. I labelled 81 points in faces A and B in a consistent manner using the same ordering of keypoints in the two faces.

The below images shows the points I labelled:


Then, I created a triangulation of these points that will be used for morphing. I used a Delaunay triangulation since it does not produce overly skinny triangles. I computed the Delaunay triangulation on the midway shape (i.e. mean of the two point sets) to lessen the potential triangle deformations.

Computing the "Mid-way Face"

With the triangulation on the mid-way shape, I did inverse warpping of each triangle. For each triangle, I calculate the affine transformation from the one in the mid-way face to the corresponding one in the original image. Then I applied the same transformation to each pixels in the triangle to transform each pixel of the original image to the mid-way shape.

For colors, I inverse warp the pixels in the mid-way face back to the original image, then used bilinear interpolation to get the color from neighbors(Cross Dissolve). This is better than forward warpping as it eliminates holes in the mid-way shape.

Kevin
Mid-way face
Zoe

The Morph Sequence

With the ability to create an intermediate image with two sources images, we can generate a sequence of images(frames) that together become a gif.

Specifically, we have two new variables called warp_frac and dissolve_frac, controlling shape warping and cross-dissolve, respectively. In particular, two source images are first warped into an intermediate shape configuration controlled by warp_frac, and then cross-dissolved according to dissolve_frac. For interpolation, both parameters lie in the range [0,1]. They are the only parameters that will vary from frame to frame in the animation. For the starting frame, they will both equal 0, and for the ending frame, they will both equal 1.

this photo moves

The "Mean Face" of a population

In this part, we computed the average face shape of all the men in Danes facing the front of the camera. We have enough faces(33) to make sure the result is interesting. We first morphed every men's faces into the average shape. Then compute the average face of all men by taking average of all the morphed faces

Below are some examples of morphed faces, as well as the average face.


Average face
man 5 original
man 20 original
man 31 original
Average face
man 5 morphed
man 20 morphed
man 31 morphed

I also tried to morph my face into the average face's geometry, as well as the other way around.

First I identified correspondence between my face and the average face. Then similar to how I morph an image in the previous part, I find triangulation in the average face, then I find the affine transformation that transform from my face to the average face for each corresponding triangles. I used the same transformation to transform all the points inside the triangle. Then I used inverse warpping to interpolate the color using cross-dissolve.

I did the same thing to morph the average face into my face. The difference is that we find triangulation in my face and find affine transformation from the average face to my face.


my face
average face
my face to average face
average face to my face

Caricatures: Extrapolating from the mean

I also produced a caricature of my face by extrapolating from the average face above. This is done by adding my points by a scaled difference between my face and the average face. Then morph my face to the new points. This will exaggerate the difference between my face and the average face.

As seen below, I have larger nose and thicker lips than the average Danes. These traits become more obvious in the caricature.

caricatured me

Bells and Whistles: Changing Gender of Image

For Bells and Whistles, I worked on changing the 'gender' of my face. I first found an image of the average women from Google. Then I morphed my face to the desired mean face by just the shape, just the appearance, and both. For morphing by just shape, no cross-dissolve is done. The color is taken from my face only. We can see that my nose is smaller. For morphing by just appearance, no warpping is done. I only cross-dissolve between the average women image and my face. We can see that I have much whiter and smoothier skin.

As we can see below, morphing with both shape and color gives the best result. cross-dissolve makes my skin whiter and smoothier, which are features more related to females. Also, morphing by shape makes my eyes larger, canthus slightly upwards and nose smaller, which are also characteristics that are more common on females' faces. Indeed, my face looks more femine after the morph.


my face
average women face
morph shape
morph both
morph appearance