CS 194 Fall 2020

Project 3: Face Morph

Andrew Aikawa

Overview

In this project, my goal is to create a smooth animation of morphing my face into someone else's face. I also compute the mean face of a population to interpolate/extrapolate my face from the mean

Defining Correspondences

First we have to define our keypoints. I did this manually using `plt.ginput`. After getting a set of points that I was satisfied with I saved them to a file

My face annotated
Mei Nagano's face annotated

Computing the "Mid-Way Face"

With the same set of correspondence points I can create the middle face by first defining the keypoints of the midface, which is just the pointwise average of the two input faces. From there, we can color the mid face by first calculating the affine transforms from the mid-way triangulation to each of the inputs and coloring the triangles by sampling the input face "texture" through interpolation based on the location of the affine mapped pixel. For the mid-way face we just take the average of the two colors sampled between the inputs for a given pixel in the midway face

Mid-way face

Morph sequence

We want a smooth animation between the two inputs. To accomplish that I linearly interpolate between the structures of the two inputs as well for the color of the two inputs

Me to Mei Nagano

Mean Face of a Population

We can compute the mean face of a population by repeating the process for the last 2 parts except instead of having 2 input faces we'll have n in general. This instead means that each individual face has a weight of 1/n instead of 2. We use this new weight to calculate the mean structure and mean coloring. I used the data from here

Below I display the average face and some individual faces from the data set that I change the structure to the average

Population Average Face
I also morphed my face into the average structure and vice versa
Me to Average
Average to Me

Caricatures: Extrapolating from the mean

To extrapolate from the mean, instead of picking an interpolation value in [0, 1], I pick `1.5`. I did this for a population of smiling faces which after extrapolating my face to the smiling population average, my portrait goes from neutral expression to exagerratedly smiling

My extrapolated smile

Bells and Whistles

For my bells and whistles, I just morphed between band members of the world famous kpop group TWICE

Lessons

I learned that my face is disproportionate and asymetrical