CS194-26 Project 3: Face Morphing (Ujjaini Mukhopadhyay)
Overview:
In this project, I learned how to morph between two faces (or sometimes more). Morphing inovlves two transformations -- a warp and a cross-dissolve. The warp allows us to change the shape of a structure whereas the cross-dissolve is some kind of weighted average of the interpolated colors. The warp must be computed BEFORE the cross-dissolve because the colors involved from the cross-dissolve depend on how each pixel is transformed from starting image to ending image.
Computing the Warp
In order to compute the warp, I must define a bunch of correspondences. From here, I build a Delaunay triangulation which is a balanced triangulation. The reason I want to triangulate is so that I can define affine transformations from starting image to ending image for each triangle.
Computing the Cross-Dissolve
To color the shape transformation, we use a weighted average of the pixels from the two images. However, to know what pixels, we use the transformation defined above. If we use the forward transformation, we end up with holes in the resultant image because every pixel in the resultant image does not necessarily map to a specific pixel in the beginning images. Thus, it is better to calculate the INVERSE transform and map pixels in resultant images to beginning images. Sometimes, these will cause fractional pixels in beginning pixels so using some kind of interpolation function will allow us to approximate the best possible values.
Defining Correspondences
I'm starting with the following two images:
After defining their correspondences and their middle points, I can draw the following Delaunay triangulation:
Computing the Midway Face
The midway face is when both original images are warped to the average of the points and thus the resulting Delanauy triangulation.
To do this, I compute the inverseAffine Transformation
Here are my results:
As you can see, the hairstyle is a little odd because Brad Pitt has blond hair and S.R.K. has black, but other than that, it seems that the face is quite believable as a separate human being! The black border exists because the images are not exactly the same size.
The Morph Sequence
Instead of equally weighting both images, I can simply use a weighted average. Since I wanted 45 frames, I can create 45 distrbuted weights between 0 and 1 (and 1 and 0 for the other image) for the warp function and cross_dissolve. Here are my results, once again between Brad Pitt and SRK!
The "Mean face" of a population
To calculate the mean face of the population, you first find the average of all the correspondences, define the triangulation, find the warps to all of original images to the average shape, and finally, cross-dissolve.
Here is the average shape and delaunay triangulation of the average shape of the Danes population:
Here are some examples of warped faces from the Dane population to the Average Shape:
Computing the average of all of these images, I get:
Warping my face shape to the average dane face shape, I get:
Warping the average face to my face shape, I get:
It's possible I got such interesting results because of some error clicking, but overall, I think it's safe to say, I'm not the best-looking dane :)
Caricatures: Extrapolating from the mean
I can take the difference between myself and the average dane's points to create a vector. Moving along this vector, I can create different cariacatures:
Here is what happens when I travel 1.5x along this vector:
I can also travel backwards along the vector. Here is what happens if I travel -0.5x along the vector. (This is making me anti-dane!)
Bells and Whistles
Converting Race
I took a famous Indian Actress (Kajol) and made her chinese. Here are the stock images that I am using:
Then I morphed just the shape, appearance, and both.
She looks Asian now!
Morphing Music Video
I made a video of me growing up and morphing from age to age! Here it is!