Project 3: Face Morphing

Aaron Sun 3033976755 Fall 2021

Defining Correspondences

We start off with the following two images. The right image is me and the left image is my roommate Kamran. Funnily, we're both mid-blink in these photos. I chose these since our facial expression is about the same including the blink.

Technically we can just do the transformation on these images, but we cropped them first in order to save time later on.

Then we wrote a tool to let us pick points on each image using plt.ginput. Using the mean of these points, we can make a Delaunay triangulation using scipy's Delaunay on these points to make a mesh. This result is shown below:

Computing the Midway Face

Affine Transformation from Triangle 1 to Triangle 2

For each triangle pair given by (A1, B1, C1) and (A2, B2, C2), we can compute a transformation matrix T which maps triangle 1 to triangle 2.

We do this by first generating T1 which maps (0, 0) to A1, (0, 1) to B1, and (1, 0) to C1. Then we get the same matrix but for triangle 2, given by T2.

Then our transformation is given by T2 x T1^-1. Any point in triangle 1 gets mapped to the B1 C1 basis and then that is mapped to triangle 2.

Warping and Dissolving

Now suppose we want to find some intermediate face between the extreme faces. We can define the amount of each face using two factors, alpha and beta.

Alpha will determine the amount of morphing which is done between the face shapes. So alpha = 0 will be the points defined on face 1, alpha = 1 will be on face 2, and alpha = 0.5 will be somewhere inbetween.

Beta will be the cross dissolving factor, which determines which pixels from which image (or a weighted average of the two) are used.

Warping

Suppose for our two triangles we have affine transformation T. Then find a mapping from points in triangle 1 to the intermediate triangle with parameter alpha, we use T1' = I * (1-alpha) + T * alpha, where I is the identity. That is, we use (1-alpha) of the original triangle and alpha of the transformed triangle. We can use a similar process for triangle 2.

Interpolating

Now suppose we want to put beta amount of the points from triangle 1 onto the intermediate triangle.

We first get the transformation T1' from the previous section, and we use the inverse of that to find the points in the original image which correspond to each pixel in the intermediate triangle.

These points may not be exactly integer, so we can interpolate the original image and use those interpolated points. Then we have the values from the original image which go in to the intermediate triangle, and we can multiply by beta at our leisure.

Midway Face Reveal

Setting alpha = 0.5 and beta = 0.5 gives us the following image:

The Morph Sequence

We can extend this idea to making a video, where alpha and beta both vary at the same rate from 0 to 1. Let's try that:

Pretty cool!

The Mean Face of a Population

Using the database from this link, we were able to find the average face shape of a population.

Here are some of the faces in this dataset, morphed to the average shape. Note the images are brighter since plt automatically saves the images to be normalized.

Original Image	Morphed Image

And finally, we can average all of these morphed images to get a population mean:

Now we can morph my face to the mean shape as well. The left is the original and the right is the morphed version.

And finally, the average face morphed to the same shape as mine:

Caricatures

We can also use this population mean to get a caricature of myself. We do this by setting the alpha from before to be negative.

Bells and Whistles: Changing Ethnicity/Gender

Using images from the World of Averages, we can change the gender/ethnicity of my face. Let's give that a try.

Me and the Average French Woman

Here are the two images we will use:

Changing just the shape of my face to fit the average french female:

It looks like my eyes got bigger and my face got longer. Now trying to change the average french female to my face shape:

And finally a full morph:

Me and the Average Russian Man

Doing the same thing for the average Russian man.

Changing just the shape of my face to fit the average Russian man:

It looks like my eyes got bigger again but my face looks... rounder? Now trying to change the average french female to my face shape:

And finally a full morph: