Project 3: Face Morphing

Taige Hu

This project, we dived into the process of face morphing, using triangulation to project pixels of one image onto the shape of another and vice versa.

For the initial face-morphing, I decided to morph my face into that of George Clooney's, images are below

Myself (A)	Mr. Clooney (B)

Defining Correspondences

I used the matplotlib ginput functionality to pick 45 points on both of the images that correspond to important facial features such as eyes, nose, eyebrows, etc. The correspondence points are shown here:

Myself (A)	Mr. Clooney (B)

Then, I found the triangulation of these points using the python library function scipy.spatial.Delaunay. Rather than using the points for either image A or B, I found the halfway point between each corresponding coordinate and triangulated using those. This helps to reduce the occurence of deformed triangles.

Computing the "Mid-way Face"

An important aspect of this part is the Affine transformation matrix, which is a one-to-one transformation from one linear coordinate to another. For our triangles. we find the Affine transformation that transforms from the vertices of one triangle to the other, matri A such that

$$AV = V'$$

$A$ can be solved for by algebra, $A = V'V^{-1}$.

Using the Delaunay triangulation from the halfway shape, I iterated through all of the triangles. In each triangle in this halfway shape, I calculate the affine matrix that transforms both image A and B's cooresponding triangles to the halfway's triangle. I then use skimage.draw.polygon to find all the pixel indices that are inside each in the midway shape triangle. With the affine matrix, I invert it to perform an inverse warp on all the indices inside the triangle from the halfway shape to the original images to find the correct pixel values inside the halfway image.

Finally, I combine the pixel values for A and B with the cross-dissolve method.

Myself (A)	Mr. Clooney (B)	Halfway

The Morph Sequence

As we change the weights for cross-dissolve and warp step-by-step, we can achieve a video of the morph completely from image A to image B in gif form. To create the gif, I obtained 45 image frames through incrementing the alpha values, and then merged them using an online resource giphy-maker.com.

The "Mean face" of a population

I used the FEI database to calculate the mean face of all-gender smiling faces using their images and annotated points.

Here are some examples of faces morphed into the average shape:

Original	Morphed to average

Using the previous techniques, here's my face morphed to the average shape, and the average shape morphed to my shape:

To Average	To Me

And finally, I produced a "fun" little caricature of my face by setting alpha to a negative value and morphing my face with the mean.

Bells and Whistles

I try to give my original image a smile by morphing it with the average smiling face from the Brazilians.

Here are the results:

Shape Combine	Appearance Combine	Both

That really put a smile on my face!