In this project, we will play around with the beautiful human face. It turns out that with some clever tricks using some rather basic image transformations, we can morph the faces of humans into each other quite smoothly. Additionally, we can use this technique to explore the face of populations of humans, and see what the average human will look like!
For this project, I will be morphing my face into that of Mika, one of my favorite singers, and probably the most talented. His vocal range is absolutely insane, and he speaks sings songs in fluent French and Italian on top of English. I will also make a face morph music video that matches the lyrics to his most popular song 'Grace Kelly' at the end.
Zach Wu (me!)
Mika
Defining Correspondences
The first thing we would like to do to is to map the various facial features between me and Mika. This will tell what points are consistent between the two faces, and what parts of the face needs to be morphed to where.
To accomplish this, I used matlab's very handy cpselect function, and manually selected the correspondences. I matched up key features between the two faces, including the eyes, nose, mouth, and jaw.
After we have the points, we must connect all the points intro triangles to use when transforming. We will warp each of these triangles individually using an affine transformation, allowing us to alter the face shape of the images. Using Delaunay triangulation, we end up with the following results, with triangles in each face having a corresponding counterpart on the same facial features.
Computing the mid-way face
Having now defined the correspondences between the two faces, we can now begin to try morphing. The first thing we will try is creating a midway face.
First, we compute the average face shape by averaging the correspondence points on the two faces. Using this, we will compute an affine transformation (using linear algebra) for each triangle, and then transform that triangle's pixel to match the new shape. Doing this for each face allows us to warp it to the average shape. We then overlay the two images to get the midway face.
zach warped to avg shape
mika warped to avg shape
mid way face
The Morph Sequence
Now we are ready to complete the entire morph sequence. We can simply repeat the steps from above in finding the midway face, but rather than going midway, have different weighting for each images. We range the amount to the face shape each face contributes, and the opacity value, and range that from 0-1 to get the entire sequence. After doing this for 100 frames, we end up with a smooth* transition from me to Mika and back. Face morph accomplished!
* I did mess up in the eye section, resulting in an imperfect match. Will make sure to match pupils in subsequent morphs. The morph overall is smooth though!
The "Mean face" of a population
.Let's now use or warping and morphing techniques to explore a larger population of human faces. Using this Brazilian Face Database, I sought to create the average face.
Using provided points, we calculate the average shape of the population, to which we will morph all faces in the population too.
before
avg shape
before
avg shape
before
avg shape
Doing the same thing as above for all 200 faces in the dataset, we then overlay all of them and average out the pixel values to get the average face. There are also smiling images, so we can also get the average smiling face.
the averaged face
the averaged face smiling
Now we can warp us to the average face, and the average face to our face shape to see what that looks like
My face in the average face shape
avg face in my face shape
Caricatures: Extrapolating from the mean
With the mean face computed, we can actually calculate how our face shape differs from the average mean. We can then exaggerate these differences to create a caricature, which theoretically exaggerates the unique differences of my face. We can use different values of alpha to dial in how much we exaggerate the differences. In morphing, we use alpha between 0,1. but to create the caricature, we will use alpha above 1 to create the caricature.
alpha = 0
the avg population shape
alpha = 1
my normal face shape
alpha = 1.5
alpha = 2
alpha = 4
Bells & Whistles: Changing Gender
Using face morphing, let's see how we can make my face more feminine. We do this by getting an average female face, and changing my shape, appearance, and both to be more like the avg face. This may give an appearance of making my face more feminine.
me
average female face
adding on avg female appearance, keeping my original face shape
my face warped closer to the average female face
both shape and appearance close to average female face
Bells & Whistle
Here's the music video I promised in the introduction. The song is about Mika's frustration trying to change his identity after meeting with record executives. He goes through many identity changes in the lyrics, which we will morph!
Sorry if there are ads btw. the song is copyrighted.
Bolded items below are what we morph to, starting from mika.
Lyrics: I tried to be like Grace Kelly But all her looks were too sad So I tried a little Freddie (Freddie Mercury) I've gone identity mad! (Mad, mad, mad!) I could be brown(gingerbread man), I could be blue(blue man group), I could be violet(Violet Parr) sky I could be hurtful, I could be purple(idk what this is), I could be anything you like Gotta be green(grinch), gotta be mean, gotta be everything more Why don't you like me? Why don't you like me? Why don't you walk out the door!
I think the fact that I had to manually do correspondence not only made the music video incredibly tedious, but it also made some of the morph results suffer as there were errors in how I made the correspondence. In the future, one avenue to explore is automatic selection of key facial features using a neural network or something.