CS194-26 Project 3: Face Morphing (Ujjaini Mukhopadhyay)

Overview:

In this project, I learned how to morph between two faces (or sometimes more). Morphing inovlves two transformations -- a warp and a cross-dissolve. The warp allows us to change the shape of a structure whereas the cross-dissolve is some kind of weighted average of the interpolated colors. The warp must be computed BEFORE the cross-dissolve because the colors involved from the cross-dissolve depend on how each pixel is transformed from starting image to ending image.

Computing the Warp

In order to compute the warp, I must define a bunch of correspondences. From here, I build a Delaunay triangulation which is a balanced triangulation. The reason I want to triangulate is so that I can define affine transformations from starting image to ending image for each triangle.

Computing the Cross-Dissolve

To color the shape transformation, we use a weighted average of the pixels from the two images. However, to know what pixels, we use the transformation defined above. If we use the forward transformation, we end up with holes in the resultant image because every pixel in the resultant image does not necessarily map to a specific pixel in the beginning images. Thus, it is better to calculate the INVERSE transform and map pixels in resultant images to beginning images. Sometimes, these will cause fractional pixels in beginning pixels so using some kind of interpolation function will allow us to approximate the best possible values.

Defining Correspondences

I'm starting with the following two images:
Brad Pitt
Shah Rukh Khan
After defining their correspondences and their middle points, I can draw the following Delaunay triangulation:
Delaunay Triangulation

Computing the Midway Face

The midway face is when both original images are warped to the average of the points and thus the resulting Delanauy triangulation. To do this, I compute the inverseAffine Transformation Here are my results:
Midway face of Brad Pitt and Shah Rukh Khan
As you can see, the hairstyle is a little odd because Brad Pitt has blond hair and S.R.K. has black, but other than that, it seems that the face is quite believable as a separate human being! The black border exists because the images are not exactly the same size.

The Morph Sequence

Instead of equally weighting both images, I can simply use a weighted average. Since I wanted 45 frames, I can create 45 distrbuted weights between 0 and 1 (and 1 and 0 for the other image) for the warp function and cross_dissolve. Here are my results, once again between Brad Pitt and SRK!
Morph between Brad Pitt and SRK

The "Mean face" of a population

To calculate the mean face of the population, you first find the average of all the correspondences, define the triangulation, find the warps to all of original images to the average shape, and finally, cross-dissolve. Here is the average shape and delaunay triangulation of the average shape of the Danes population:
Average Danes Face Shape
Here are some examples of warped faces from the Dane population to the Average Shape:
Warped
Warped
Warped
Warped
Computing the average of all of these images, I get:
Average Danes Face
Warping my face shape to the average dane face shape, I get:
Average Danes Face
Warping the average face to my face shape, I get:
Average Danes Face Warped to my Faceda
It's possible I got such interesting results because of some error clicking, but overall, I think it's safe to say, I'm not the best-looking dane :)

Caricatures: Extrapolating from the mean

I can take the difference between myself and the average dane's points to create a vector. Moving along this vector, I can create different cariacatures: Here is what happens when I travel 1.5x along this vector:
Cariacature towards Danes Face
I can also travel backwards along the vector. Here is what happens if I travel -0.5x along the vector. (This is making me anti-dane!)
Cariacature away from Danes Face

Bells and Whistles

Converting Race

I took a famous Indian Actress (Kajol) and made her chinese. Here are the stock images that I am using:
Kajol
Average Chinese Woman
Then I morphed just the shape, appearance, and both.
Just Appearance -- not even lined up!
Just Shape
Shape and Appearance
She looks Asian now!

Morphing Music Video

I made a video of me growing up and morphing from age to age! Here it is!