Face Morphing

CS 194-26 Project 3    Shivam Singhal    October 6, 2021

---

For this project, we were tasked with creating a "morph" animation of our faces into someone else's face, computing the "mean face" of a population, and using that average face to create a caricature of ourselves.

---

Part 1: Defining Correspondences

For this part of the project, I relied on python's ginput command in order to manually select keypoints on important facial features that would allow me to morph between different images. I chose to select about 40 keypoints in both my starting and target images. While I do recognize that we were supposed to use our own photos as the starting point, I thought it would be more interesting to see George Clooney (photo from assignment website) turn into someone else. And that someone else was Former President Obama (photo found here: https://www.artsy.net/article/artsy-editorial-how-artists-captured-president-obama-legacy). Let's see one silver fox turn into another! Below are the keypoints I selected.

I then took the average of the two correspondence sets and calculated the Delaunay triangulation, which I overlayed on both of my images and displayed below.

Part 2: Computing the 'Mid-Way' Face

In order to generate the mid-way face between Clooney and Obama, I computed their average shape by taking the average of their keypoints. Next, I performed the inverse warp for both of the images into this average shape by first computing the transformation matrices that map us back from the average image we calculated to the original images. I used the triangulations generated in the previous step to calculate the affine transformation between these images, which gave me the needed transformation matrices for the warp. Bilinear interpolation using RectBivariateSpline was lastly performed to get the average color between the images properly assigned. The mid-way face between Clooney and Obama, as well as the original images used, are displayed below.

Part 3: The Morph Sequence

In order to see Clooney morph into Obama in real-time, I produced a morph sequence. Between the original Clooney and Obama images, I performed a morph using the keypoints and triangulations that I defined in the first part. What I changed between each of these morphs was two parameters: warp_fraction and dissolve_fraction. Warp_fraction controls how much the triangles in the original images contribute to the average shape, and then, the images are cross-dissolved based on the dissolve fraction. To generate the frames of the displayed gif, I had my warp and cross-dissolve fractions range between 0 and 1/45. Here is the link to the youtube video: https://youtu.be/XqxVzWeGWrY. Below is a gif as well:

Part 4: The 'Mean Face' of a Population

For this part of the project, I chose images featuring males with full frontal 'happy' expressions from the Danes dateset; there were 33 images in this dataset. First, I parsed the keypoints for these images that the researchers selected from the .asf files provided. I then computed the average face shape by taking the mean of the corresponding point sets and generaing a triangulation of the resulting average points. Next, I morphed each face of the dataset into the calculated average shape using the previously written inverse warping function. Below are some examples of these morphs.

I finally computed the average face of the chosen population by taking the sum of all of the morphs and dividing this sum by the total number of images. Below is the average face:

I used a new image of George Clooney for this part of the project that is shown below:

Below is Clooney's face warped into the average Dane smiling male geometry:

Below is the average Danish face warped into Clooney's geometry:

Part 5: Caricatures

Using the mean face that was generated in the previous part, I generated a caricature of Clooney's face through extrapolation in terms of the difference in shape and color. Below is the caricature that I generated:

Bells and Whistles 1: Changing Gender

I decided to change the gender of Mr. Clooney because he is my best friend, and for this I used the female images from the Danish dataset. I calculated the average shape and face of this dataset, and I morphed Clooney's shape, Clooney's skin color, and both. Below are the individual shape and appearance morphs along with both shape and appearance morphs:

Bells and Whistles 2: Growing up!

I finally decided to show my face! Below are a few images of me growing up morphed together. The first image is from my trip to the Taj Mahal, the second image is from my favorite festival, Diwali, and the third image is from after a gruelling tournament...speech tournament! Here is the youtube link: https://youtu.be/MLKzbSWSk3Y