Project 4: Face Morphing

Student Name: Divyansh Agarwal (cs194-26-aba)

Course: CS 194-26 (Computational Photography)

Background

In this assignment, we try to morph one face into another. This involves morphing both shape and appearance.

Defining Correspondences

I want to morph my face into the face of one of the best Indian cricketers, Rohit Sharma. In order to this, I first had to define correspondences, and these are shown below for the images:
Forest

Correspondences for my Face

Forest

Correspondences for Rohit Sharma's Face

Mid-Way Face

I computed the average of the correspondences defined for each of my images, and warped each image into the set of mean coordinates. I then cross-dissolved the morphed images to get my 'mid-way face'. I decided to morph my face into the face of one of the best Indian cricketers, Rohit Sharma, and the mid-way face can be seen below:

Snow Me

Forest Mid-Way Face

Forest Rohit Sharma

The mid-way face looks very good! You can see that the shape has been morphed, as my nose is a bit tilted and my face is a bit asymmetric, but the mid-way face has a more straight noise and a symmetric face. The face is also slightly fatter than my face and a bit thinner than Rohit Sharma's face. The appearance has also been morphed, as the skin shade and texture for the mid-way face is somewhere 'in between' of both the images.

Morph Sequence

I generated a sequence of the morph images, where I took a weighted average of the shape and appearance, with the weights for each image changing as the sequence progressed. I used the same set of weights for averaging appearance and shape for any given image, to simplify my computation. I started off with giving all the weight to my image, and then slowly decreased the weight given to my image, and increased the weight given to Rohit Sharma's image as I went further along the sequence. The weights for both images always added up to 1.

The morph sequence can be seen below:

via GIPHY

Average Faces

I used the initially released subset of 37 images of the Danes face datase, and decided to use the subset of males in this dataset.

I computed the average face of the males in the Danes dataset, as shown below:

Forest

Average Dane Male

I also computed the average face of all people in the Danes dataset, as shown below:
Forest

Average Dane

The average face is not very different from the average male face, as there were very few females in the dataset I used. However, we can observe some subtle differences, such as the moustache of the average face being lighter than the moustache of the average male face.

I morphed each male face in the Danes dataset onto the average male coordinates, and some examples are shown below. Some morphed images look reasonable, while some morphed images don't look realistic and don't look pleasing.

Example 1

Forest

Morphed Face (Looks Reasonable, although with some issues)

Example 2

Forest

Morphed Face (Looks Reasonable)

Example 3

Forest

Morphed Face (Not reasonable, not aesthetically pleasing!)

I morphed my face onto the average male geometry, as shown below:

Forest

My Face Morphed Onto Average Male Geometry

The morph does not look very good, for multiple reasons: the mean coordinates did not have any correspondences specified for the forehead, so my forehead does not align with the mean Danes male forehead. There is also a big difference between facial geometry and the geometry of the mean Danes male face, so the morph did not look great.

I also morphed the average male face onto my geometry, as shown below:

Forest

Average Male Face Morphed Onto My Geometry

The morph doesn't look good, again because of the big difference between facial geometry and the geometry of the mean Danes male face

Caricatures

I created caricatures of my face by accentuating the difference between the coordinate correspondences for my face and the average male face. I did this by taking the difference between my geometry and the average male geometry, multiplying the difference by a factor, and then adding it back to my cooordinates for the correspondences. Then, I morphed the new coordinate correspondences for my face on the average correspondences. I tried multiplying the difference by factors of 0.2, 0.3, 0.4, 0.5, 0.6, and the results are shown below:

Snow

Factor: 0.2

Snow

Factor: 0.3

Snow

Factor: 0.4

Snow

Factor: 0.5

Snow

Factor: 0.6

Bells and Whistles

I morphed my face onto the average Spanish female face, which I downloaded off the internet. The mid-way face below looks good!

Snow Me

Forest Mid-Way Face

Forest Average Spanish Female Face

I also tried morphing just appearance and shape, with the following results. Clearly, just morphing based on appearance (i.e., just doing cross-dissolve) does not give a good result.

Forest Morphed Shape

Snow Morphed Appearance