Hi! This project explores affine transformations as a means of morphing faces together, finding the average face from a population, and caricaturing the unique features of a population or individual.

  • Part 1: Morphing Two Faces Together

  • Part 2: The "Mean face" of a population (includes Bells & Whistles!)

  • Part 3: Caricatures (includes Bells & Whistles!)

Part 1: Morphing Two Faces Together

Creating a morph animation consists of three steps. First, I defined corresponding points that serve to define the shape of a face. I labelled these faces by hand in order to most accurately identify similar locations on a variety of faces.

Example 1: Jacob and Prangan

Before starting with my face, I tried creating my morph code on two of my friends, Jacob Schillen and Prangan Tooteja. The two have similar facial hair, but have different ethnicities, skin tones, and feature shapes, so it made for quite an interesting warp.

After defining correspondences, I could display a triangulation of the shape points I'd defined on each face. I chose to use the average shape points vector between my two images and the Delaunay triangulation, which is included in scipy's spatial package. This ensured that the triangles I was warping weren't too skinny-- instead, they each cover a broader area of the face. The intention here is that corresponding triangles cover similar areas of the face.

To warp each face to an average, I first found the average of the shape points vector. Then I found an inverse warp between each triangle within this average points triangulation and the corresponding triangle within the source image. This allows me to warp both Prangan's and Jacob's face to the average shape of their two faces.

From there, the morph sequence averages their faces using a cross-dissolve parameter, initially set to 0.5 to find a perfect average. The result, showing initial images, warped images, and the morph image is shown below, along with a gif between the two faces.

Example 2: Shreya and Sravya

From there, I wanted to test the morph on my own face. I realized that my face doesn't lie on the same subspace as that of Jacob's and Prangan's due to their facial hair, so I decided to morph myself to my roommate, Shreya Mohanty. I found it difficult to get our hair to morph well, as my hair is much shorter and poofier than hers. In addition, I noticed some artifacts from the different locations of our pupils-- mine are looking up a bit more than hers in the original image.

In an attempt to fix the hair overlap, I decided to tie mine back and redo the morph image. I wasn't able to grab another image of my roommate, so I think this one wasn't as successful as it could have been. The lighting also varies between these two images fairly drastically, which creates an odd shadow tone for the morphed image. On the other hand, the first morph was taken in the same room under the same lighting, giving the skin a much more natural look.

Example 3: Sravya and Jacob

I then attempted the morph I knew wouldn't work too well-- my extra head hair with Jacob's extra facial hair. The end result is a rather cute elf. The artifacts come from likely mislabelling around the ears-- my initial labelling scheme on the two men placed a lot of points defining the ears, but I placed the points around my ears fairly arbitrarily in the general location of where my ears would be under my hair.

To fix this, I created a morph of the image where my hair was tied back. This one worked a bit better, but the facial hair makes it tough to morph together.

Part 2: The "Mean face" of a population

The next challenge I took on was to morph together to find the average of 40 Danish faces in a population. I used images from M. B. Stegmann, B. K. Ersboll, and R. Larsen. FAME -- a flexible appearance modelling environment. IEEE Trans. on Medical Imaging, 22(10):1319{1331, 2003}. These images had pre-labelled correspondence points, which was different from the labelling scheme I used earlier.

However, I was able to use the pre-existing labels to find the average shape of the Danish face. I first started by finding the average shape vector, and then morphing the appearance of each face to this shape. Examples for 10 of the faces are shown below.

From there, I found the average of these appearances to recreate the average Danish face.

After, I asked what I would look like if I were Danish-- by warping my face to the shape of the average Danish face. I did the same for the average Danish face, which made the face much rounder. This was done by collecting the points for each face and warping the other to that shape.

After, I asked what I would look like if I were Danish-- by warping my face to the shape of the average Danish face. I did the same for the average Danish face, which made the face much rounder. This was done by collecting the points for each face and warping the other to that shape.

Bell and Whistles: Ethnic Warps

In this part, I decided to warp my face to the average female face from a few different ethnicities. What I found interesting is that the warp to my own ethnicity stands out quite a bit-- while it made my face broader, it preserved my eye and lip shape much more. I used this to inspire another Bells and Whistles caricature further below!

Part 3: Caricatures

To make my individual features stand out, I caricatured my features away from the average Danish face. This was done by multiplying my facial shape by a factor of 1.5, and subtracting 0.5 times the average Danish face shape points. This creates a face shape that extrapolates away from the average Danish face and closer to my own! It made my lips fuller, eyes bigger, and face much rounder. It also emphasized the assymmetries in my face and in the image.

Then I did the same for the Danish face, which created thinner lips, smaller eyes, and a longer face.

For fun, I decided to create a spectrum displaying a range between caricatured faces with the morphed face in the center.

Bell and Whistles: Ethnic Caricatures

I figured that caricaturing my face away from the average Danish face was going to amplify many features that were not only unique to myself, but to my ethnicity. The next question I was curious about was whether I could caricature my face by extrapolating away from an average Indian female face.

I noticed that unlike the caricature based on the Danish average, this one showed minimal changes. I found this extremely interesting, as it affirmed that much of the caricaturing was a result of ethnic features. I also found this interesting to relate to how we identify individuals visually-- often, it's by identifying what makes their faces unique. It's interesting to me that among a population of Indian faces, the features that are considered unique on my face might be drastically different from what is considered unique in a majority-white society like the one I grew up in.