Project 3: Face Morphing

1. Correspondences

We use 56 keypoints (excluding the four corners) as the correspondenes. The images used for morphing, the keypoints and the mesh are shown as follows:

original image triangular mesh

A problem with this mesh is that some triangle are crossing physical border, such as the region between the outer tip of the eyebrow, the tip of the ear and the temple. This will become a larger issue in the caricature section.

2. Midway Face

We can apply affine transformations on each of the triangles and then apply a linear interpolation between the two morphed image as a midway product. Below is the morph between the two set of images when the morph factor is 0.5.

Note that the morph on the hair is noticeably worse than the other part. This is due to the fact that we do not have keypoints on the hair. Even if we have, since the growth of hair is not a linear transformation, it will be hard to achieve the morph between short and long hair.

3. The Morph Sequence

Below is the morph on the two sets.

Bells and Whistles

Additionally, we apply a whole body transformation between a legit Saber figurine and the Saber figurine known as the "Evil God".

Note that the result on the right sleeve, the dress and the ribbon are significant worse than the other part. This is NOT due to a wrong set of control points. The cause of the sleeve morph error is that it is touching the dress in one of the image, and the algorithm is not able to interpolate white background from pure blue color. The other two are due to the slight difference in perspectives in the two images. This set of images contains more than 100 control points, and the difference cause some triangles to twist and overlap on itself, causing the issue.

4. Average Face

The dataset we used is a public face dataset called All-Age-Faces. The images in this dataset spans from 1-year-old to 80-year-old, and the majority are asians. As an Asian myself, it is easier to extrapolate using a dataset as the same ethnicity with me. We (me and ) took a random subset of 310 images across all ages.

The images are annotated by hand by me and Haohan Lin (course id acl). By find the average shape and morph all faces into the average shape for all images and each sub-population, we got the following average face for each group.

all images age 0-9 age 11-20 age 21-30 age 31-40 age 41-60 age 61-80
all images
female
male
my face morphed into the average shape the average face morphed into my shape

5. Caricature: Extrapolating from Mean

We can extrapolate the shape and color using the mean image from the previous section. Below is the caricature created with factor of 0.5, using mean of different population.

w.r.t. all images w.r.t. all male w.r.t. all age 20s w.r.t. all male 20s

The shape and the imperfections of the face are emphasized by the extrapolation.

However, due to the undesirable mesh described in the first section, the temple in the caricatures are noticeably angular. This problem is lessened in the last image since the difference between the photo and the mean of the small sub-population is smaller. The problem can be fixed by a manually tweaked mesh or just add more control points.

6. Bells and Whistles: Changing Characteristics

We can also change gender or age of the image by adding the difference between the target population and the source population to both the control points and the image itself.

task reference source image reference target image result
change gender to female
change age to 0s
change age to 60s

The algorithm does a pretty good job at changing the shape and features into the target population.