Project 3 Face Morphing

Nithin Chalapathi - nithinc@berkeley.edu - SID: 3032738412

In this project, we focus on manipulating and morphing faces. In particular, we morph between different faces (showing intermediate frames), compute the average Danish computer scientist, and finally wrap up by morphing my face to the average East Asian female and male faces.

Defining Correspondences and Face Morph

First, we define correspondences for the face morph. I currently live with 2 roommates and they were the faces I morphed between. The final product can be seen at: https://youtu.be/za1zvdR5h4Q

The 3 faces I used are as follows (with the triangulations):

Niki:

Me:

Max:

To create the triangulation, I used ginput from matplotlib. I created a small helper script that would display the corresponding points that need to be selected between two images given a reference set of points for one image. To my understanding, matplotlib doesn't have a cpselect like matlab and I used this as a primative alternative. The triangulation is formed from the average of the two face shapes. To perform the cross-dissolve, I used scipy's built in RectBivariateSpline. For the midway face, I used Niki and I's faces.

Midway face:

The final product, in the youtube link, is a result of combing the morphs from Niki to me, me to Max, and finally Max to Niki.

The "Mean Face" of a population

For this section, I used the Danish dataset. Here is an example of the correspondence points on the one of the images along with correspondences on my face.

Danish Image:

Me:

Similar to the face morph, I computed an average face shape using all the correspondences. I then mapped each Dane to the average shape. Here are a few examples of the results (original followed by the average shape morph).

After cross dissolving by averaging all the shape transformed Danes, the final product is

Morphing my face to the average Danish face I get:

And morphing the average danish image to my face shape:

I found the results very interesting. In my face morphed to the average danish shape, you can see that many of my facial features are significantly narrower. This is especially prevalent with my nose. Conversely, the average Dane morphed to my face has a significantly wider nose.

Caricature: For the caricature section, I exaggerated the morph of my face to the average Danish face. I used an alpha of 1.3 to produce this image. It is interesting to note the nose is even narrower in this image.

Bells and Whistles - Changing Gender

For my bells and whistles, I morphed my face to the average East Asian male and female faces I found on Google. Here are the images:

And my corresponding image (resized):

When morphing with the female face, here are the results for just the shape:

And with color:

And for the East Asian male face:

And with color:

Concluding Thoughts

I had a lot of fun with this project and even used my code for some fun results on the side using images my friends sent me! The most difficult part of the assignment was computing the transformation matrix for the affine transformation. While my math was correct, I forgot to transpose the correspondence matrices before computing the transformation matrix. While a simple error, it took a long time to debug. Besides that, the rest of the assignment went well.