Project 3: Face Morphing

Timothy Kha

1.1: Defining Correspondences

Using matplot lib's ginput, I hand-picked points on both images. The first image is my friend Nick, the second is one of his favorite Super Smash Brothers player "Light".

After picking keypoints, I averaged the two sets of key points to produce a set of average keypoints. I used the Delaynay triangulation package to compute a triangulation.

Original Nick	Original Light
Nick with triangulation on average of keypoints	Light with triangulation on average of keypoints

1.2: Computing the "Mid-way Face"

Using the average shape computed in part 1.1 (average of keypoints of each face), I then warped both faces to that shape.

This was achieved by iterating through all triangles in our defined triangulation, and then performing an inverse warp for both images. I needed to compute an affine transformation for each set of triangles, then find the interpolated values in the original image, and then finally set these values in my result. We can see below how this actually looks (Nick and Light morphed to avg shape).

Finally, in order to achieve the midface, we just average the colors of the two morphed images to produce our result.

Original Nick

Nick morphed to avg shape

Mid-way face of "Night" (Nick + Light)

Light morphed to avg shape

Original Light

Original Nick

Mid-way face

Original Light

1.3: The Morph Sequence

For the morph sequence, we combined the methods we used for the two parts above to create a morph sequence from image A to B.

We have two new variables that we need to set for each frame: warp frac and dissolve frac; both are values from [0, 1].

The warp frac is responsible for controlling the weighted average for our keypoints that we morph each image to for each frame. And the dissolve frac is responsible for the blending the colors after we have morphed both images to the weighted "average" shape for that frame

Using this, we created a 45-frame warp of Nick to Light:

Nick morphing to Light

1.4: The "Mean face" of a population

I used the database for the Danes to find the "average Dane" face. Using the pre-labeled points, I performed a similar process to part 1.2 and morphed each face in the database to this average Dane face. Here are some examples:

Original Face 1	Morphed Face 1
Original Face 2	Morphed Face 1
Original Face 3	Morphed Face 3

And here is what the average Dane looks like:

Average Dane Face

Here is what Nick looks like warped to the average Dane's geometry and vice versa:

Nick's Face	Nick to Average Dane Face
Average Dane Face	Average Dane to Nick's Face

1.5 Caricatures: Extrapolating from the mean

Using keypoints from the population mean of the average Dane from above, I performed extrapolation by chosing alpha values less than 0 or greater than 1.

This was accomplished by taking the keypoints of Nick, subtracting it from the avg_dane keypoints, scaling it by alpha, and then adding it back to the keypoints of Nick (and then morphing Nick to these keypoints). Here are some of the results: