Project 3: Face Morphing

Chenyue Cai

---

Overview

Goal: We would like to morph one face into the other and produce a gif out of the images. In that case, we would like to produce images of morphed faces on different stages. in that case, we will need to first compute the morph and, at the same time, control how much one face is morphing into the other in terms of shape and color.

Methods: Follow the following steps
"Points" -> "Triangulation" -> "Affine Transformation" -> "Loop Over" -> "Interpolate"
1. Points: I use interactive GUI from python for the user to select the points. For human faces, I default the number of points to select to sixty so that I can capture all the essential information on a human face. When selecting points for multiples objects, I keep the sequence of the points for different key face features to develop correspondence.
2. Triangulation: Given the corresponding points, I can get Delauney triangulation of one set of points. I keep this triangulation as the basis for all the other points of the different images. Therefore, I transformed all the points set into their "triangulated" forms.
3. Affine Transformation: Since we already get the corresponding triangles from the triangulation, we can transform each triangle into the shape of the other through affine transformation. I think of this as a change of basis. Given three points, I formed two vectors, and regard these two vectors as the basis. Hence, I am able to compute the change of basis matrix while adding on the translation of the origin. What I get is a 2*3 matrix. In that case, as I dot product this transformation matrix with any given points [x,y,1], I can get the transformed coordinate. One thing worth noticing is that I used inverse warping in order to make sure the points exist. This is done by finding the the corresponding point of the result from the input.
4. Loop Over: This part is easy. Looping over all the triangles is like piecing togethor all the transformed triangles into the final result.
5. Interpolate: Through interpolating on the warp fraction and the dissolve fraction, I can make the morph result more or less like one of the image or the other.

---

Part1: Mid-Way Face

Here, I morphed my friend's face into mine.

Source Image:

Triangulation:

Morphed Faces:

---

Part2: Mean Face of A Population

Here, I choose the Danish computer scientists data set. I select the points for each, get the triangulation, average these triangles and get the mid-shape for all the Danish computer scientists. By morphing all the scientists into the mid-shape one, I can get the mean face of the population by just averaging the morphed faces.

Mean Face of Danish Scientists:

My Face and Mean

Here is a photo of myself and I tried to morph the mean into my face shape. It factually looked more feminine and nice. On the other hand, I also tried the inverse, morphing my face into the mean shape. It does not look pretty. This is probably due to the facial difference between possilbe asian female computer scientist and the given danish mean computer scientist. Also, the danish mean face is more male like since there are more males.

Caricature

By extrapolating from the mean, I am able to get osme caricatures, which looks pretty ugly lol.

---

Part3: Bells and Whistles:

1. Morphing into another gender

From the previous part, I find that morphing oneself into another gender, ethnicity is pretty fun. Here, I find the average face for Japanese male actors and I want to morph my face into it.(Once upon a time,I do have the dream to become an actress.)

Here, I morphed myself into the shape and the color of the average Japanese actor. Below, I show three results, the first being the color change, the second being the change in both color and shape, and the last being the shape change.

2. Morphing music video with a theme

Here is a growing up/change of fashion Gif for myself. While growing up over the years, I traveled to different places and took various styles of photos. In this music video, I include 13 images of myself in different ages and different fashion. I took these photos in Shanghai, Berkeley, Boston, Kyoto and Paris. Here are some of the example photos included in the video.

Here is snippet of the music video in Gif version.

Here is the Full Music Video.

3. Something Fun

Here is a morph between my friends' face and anime Haikyuu's characters. We are all super into volleyball in high school and we all agree that Haikyuu is the best sports anime or at least one of the best. We are all on the volleyball team back then, me being the right wing, Ciao being the setter and Lily the setter as well. I make this morphing video in memory of all the great times we spent togethor.

Haikyuu Characters.

Some other interesting thoughts

Given more time, I would really want to do a morphing sequence between Chinese emporers and compare these appearance ot their rule. I also really want to do a morph between all the self portrait of great artists and see how they have evolved throughout times.

---

Final Reflections:

1. I find that selecting points is crucial for face morphing, Developing correspondence and not flipping the triangles is very important.
2. This algorithm does not do very well in morphing faces with different background since the triangulation may create edges.I am thinking about using different algorithm for background morphing.