Face Morphing

Calvin Grewal

Introduction

The goal of this project was to learn how to morph faces onto each other and create an animation by simultanously warping and cross-disolving the images. For my main morph, I chose to morph my own face into Obama's face.

Defining Correspondences

Selecting Points

For this part, I chose 46 points that I believed are good facial features. I wasn't sure exactly which parts of the face to choose, so I chose many points from all parts of the face. I did so by using python's ginput function, and made sure to select the points in the exact same order.

Points Chosen

Creating a Triangulation

I chose to create a Delauney triangulation using the midway shape of the keypoints I selected. The triangulation is pictured below overlayed on the average of my two images.

Computing the "Mid-way" Face

To compute the midway face I had to define a function that returned an affine warp from one triangle to another. Since we know that an affine warp has six degrees of freedom, I setup a system of equations to solve for the six parameters of the warp, and used np.linalg.solve to solve the system. Since the warp will lead to coordinates that aren't necessarily on a grid, the last part of this section was using interpolation in order to get the correct color from the original images.. Once this function was written, I was able to warp both myself and Obama to the midway shape create the midway face shown below.

Midway Face

The Morph Sequence

For the morph sequence, I followed a very similar procedure to the previous part, and did so for 45 frames of animation. The only difference was that at each timestep t, instead of computing the midway face, I morphed each image t/45 of the way, and cross dissolved appropriately.

The "Mean Face" of a Population

For this part I used the IMM Face Database of Danish researchers. I used the first image type which was full frontal face images with no rotation. The main challenge with this part was parsing the .asf files to extract the pre-defined keypoints. After some struggle, I figured it out and got the keypoints for each of the 40 images. To comput the average face shape, I simply averaged each of the keypoints of all 40 images. Next, I warped each of the 40 faces onto this average shape using the same method as the midway face. Finally, I took the mean of all 40 images to generate the mean image, which is displayed below.

In order to warp my face onto the Danish geometry, I labeled my face exactly the same way the Danish dataset did. Next, I warped my face onto the averag Danish geometry, and also the average danish geometry onto my face using the same warping process.

My face warped onto Danish geometry

Avg Danish face warped onto my geometry

Caricatures: Extrapolating from the mean

For this part, instead of warping my face onto the average Danish face, I extrapolated further to create a caricature. I experimented with many alpha > 1 to caricatures with different intensities, and I settled on using alpha = 1.5.

Extrapolated Keypoints

Extrapolated Image

Bells and Whistles - Changing color/shape of my face

For this part, I decided to change my ethnicity to Danish, change my face shape to Danish, and also both.

Danish Calvin (just color)

Danish Shaped Calvin (just shape)

Fully Danish Calvin (both color and shape)