Defining Correspondences

For defining the correspondences, I used matplotlib's ginput() function to generate a list of points, and I used the same labeling order for both images. For selecting points, I made sure the prominent features like eyes, mouth, and nose are outlined. I also made sure the head shape is selected as well. In total, 60 points were used. As for the images, I used a picture of myself and a picture of Jin that are somewhat aligned. After getting the points, I averaged the corresponding points for both myself and Jin and used that to find the triangulation using scipy's Delaunay triangulation method.

Computing the "Mid-way Face"

To compute the midway face, I first computed the midway shape, which is done by averaging the corresponding points (as I did when calulating the triangulation). Then, in order to morph a face into the midway shape, I looped through every triangle and applied an affine transformation matrix to the pixels in an inverse mapping (each target pixel find the color it should have from the original), of course doing this NOT by looping over every pixels but doing them at the same time.

Then, after morphing my face and Jin's face shape to the midway face shape, I cross-dissolved both images (average) to get the target midway face. Below I show both original faces and the midway face:

The Morph Sequence

Morphing is controlled by two parameters: warp_frac and dissolve_frac. For this part, the code is like computing the Mid-way face but instead of averaging, I use a weighted average controlled by those to parameters. First I compute the warped points by (1.0 - warp_frac) * im1_pts + warp_frac * im2_pts, then warp both images to those points. Then I cross-dissolved both warped images according to dissolve_frac.

The following sequence is generated with 46 frames, 0-45, with 30 FPS, and increasing both parameters from 0 to 1 at the same time.

The "Mean face" of a population

I used the Danes dataset, which has 37 Dane faces with labeled correspondences. I wrote a script to read asf files to extract the points. After getting all the points, I averaged the points to get the "mean shape" of the Danes, and used this shape for the Delaunay triangulation. I also added four points to each face, which corresponds to the corners.

Here is an example Dane triangulation:

Then I warped all Dane faces to the mean shape. Here are a few examples:

If I then average all the Dane faces warped into the mean shape, then I get the "mean face":

If I then do some cropping and label my face with the same correspondence points as the Danes, then I can warp between my face and the average Dane:

Here's my face to mean Dane's shape and vice versa.

Caricatures: Extrapolating from the mean

I generated a caricature of my face by extrapolating from the mean face. Typically when morphing shape, we keep the morph frac to between 0 and 1. But, if I exceed it, then we get some very interesting and exaggerated results.

Bells and Whistles: Changing Gender

I tried changing the gender of my face using an image of an average female face found online. I labeled both my image and the average female using the same points and computed triangulation as explained above.

Then, to change the gender of my face, I morphed both the same and appearance of my face towards the shape and appearance of the average female face, which is like shifting the feature space of my face towards those of females. For the morph, I used a warp_frac of 0.5 and a dissolve_frac of 0.4.

Here I show what happens if I only morph the shape, if I only morph the appearance (by morphing female to my shape and cross-disolving), and doing them both at the same time: