CS194-26 Project 3: Face Morphing

Kevin Shi

Overview

This project is a application of affine transforms with triangulation. We "morph" faces from one to another through transformation and pixel averaging.

Part 1: Defining Correspondences

Here we first select facial keypoints on our image to find the triangulation to smoothly transition between images.

First we start with our source images, Donald Trump and myself

After selecting the facial keypoints, we also use Deluaney Triangulation to create the following:

Part 2: Computing the "Mid-way Face" and The Morph Sequence

We now compute the "midway face".

To find the average geometry, we find the average of the correspondence points. Then we apply the affine transformation by using an inverse affine transformation matrix and bilinear interpolation.

To find the average face we can then simply average the pixels of these two products because their geometry should be the same

The same principle can then be applied to find the many faces in the middle to create a smooth transition between images

Here, I used steps of .03 to create 33 frames for a "morph" gif

Part 3: The "Mean face" of a population

I used the imm face database for this part of the project to compute the "average face"
@ARTICLE{Stegmann2003tmi, author = "M. B. Stegmann and B. K. Ersb{\o}ll and R. Larsen", title = "{FAME} -- A Flexible Appearance Modelling Environment", year = "2003", pages = "1319-1331", journal = "IEEE Trans. on Medical Imaging", volume = "22", number = "10", publisher = "IEEE" }

Here we can see all the faces

As well as all their annotated keypoints

And heres one close up

We can the calculate the average geometry just as we have done before, which we show here projected on top of one of the Danes

Then we can compute what the danes would look like transformed to this average shape- using the same affine transformation on triangles as before.

And on all of the danes:

We can see that some of the sanes now have foreheads that look like they've been unnaturally distorted. This is because the dataset does not give keypoints above the eyebrows, so the transform is dependent on how the triangles of the eyebrows and the top of the page are lined up, which is not predictable.

Now that we have all the Danes transformed to a common geometrical facial shape we can compute an average

Comparing myself to the Danes

We now can compare how I look compared to the danes and play with some morphs to see some interesting results

We can then change myself into the average Dane geometry and vice-versa

We can also make a caricature of myself by finding the differences between my geometry and the average Dane geometry and add that back onto my image to exaggerate my features.

Uh-oh, might be a little racist here :). Really makes my eyes squintier and my head wider and more square

Part 4: Bells n' Whistles

We are going to take these two images and try something fun!

Let's first turn the female into my geometry

Uh-oh, that is one huge chin. Turns out I don't make a great looking female :(

Let's see what I would look like as a female

Hey, its not too bad. But nothing 5 minutes in GIMP (or photoshop) can't improve

#loveurself #nofilter

Conclusion

Although marking the keypoints by hand was very tedius, overall I had a lot of fun with this project, and seeing the morphs come to life is pretty entertaining.