Project 3:
Face Morphing

I used a combination of the starter code for aligning images from Project 2 (which uses ginput()) and Photoshop to align my two images (Simu Liu and Chris Hemsworth) appropriately. Then, I defined 62 points (9 for the hair/head, 4 for the nose, 6 for the eyebrows, 7 for the mouth, 8 for the eyes, 10 for the ears, 8 on the face outline, 2 for the neck and 8 on the image borders. With the mean of the two sets of keypoints, I provided a Delaunay triangulation to use for all morphing with the two images.

To compute the mid-way face, you must do 3 things: (1) compute the average shape, (2) warp both faces into that shape, and (3) average the colors togethers. The first step is very simple as you can just take the average of the two sets of keypoints from the two faces. The second step is more difficult. For every triangle from the triangulation done before, you must compute the affine transformation from an original face to the average shape through linear regression or other methods. This should be repeated for the other original face. Then, you can use the inverses of the calculated affine matrices for both faces to find the relevant points in the original images, and use interpolation (e.g. bilinear) to find the correct colors. These colors should then be averaged and mapped back to the warped shape to complete the morph. In practice, there are some steps that can be done differently, but that is the general gist of morphing.

The full morph sequence follows the same guidelines set in the previous section, but you can now use different ratios of warping and cross-dissolving to achieve different levels of morphing. The gif below has 45 frames and is 30 fps.

I obtained 37 images from a subset of the IMM Face Database and computed the average shape of the faces (using their annotated keypoints). I also added corner points to see the whole image after warping into the mean shape. Here are a few examples.

I took the warped faces and averaged their colors to get the mean face.

I warped the mean face to my image, and vice versa. Here are the results.

To create caricatures, I took the difference between the points for the mean face and my image, scaled it by alpha, and added my points. Here are different results.

Here I wanted to change Simu Liu to an average white American woman.

I tried out different shape morphs.

For the appearance, I didn't think it was interesting to try multiple, so I took the average cross-dissolve.

Here are the final morph results.