Overview

In this project, I created a morph animation between George Clooney and Robert De Niro using cross-dissolve/interpolation, Delaunay triangulation and affine transformations. I also utilized those methods to compute the "mean face" of a population (using the Danes dataset) and extrapolating from it in order to produce a caricature of Robert De Niro. I finally used the average face of a South African man to "change" the ethnicity of George Clooney.

Defining Correspondences

In order to define the correspondences between two images, I utilized ginput and manually selected 30 points such that there was a 1:1 correspondence between the images (e.g. George's pupil center coordinate had the same index as Robert's). Using those points, I then used Delaunay triangulation on the mean of the two point sets-- this is important as it maintains the same triangulation throughout the morph.

Computing the "Mid-way Face"

In order to compute the "mid-way face" between George and Robert, I had to compute the average shape, which involved taking the average of each keypoint location in the two faces, warping both faces into that shape, and averaging the colors together. In order to warp the faces into the average shape, I implemented an affine warp which was applied to each triangle in the triangulation from the original Robert and George images into the new average shape. This involved computing an affine transformation matrix T between each pair of original shape triangle and the corresponding average shape triangle. Here is the linear system that I set up to compute the values in the affine transformation matrix:

Then, we can use the set of transformation matrices to compute an inverse warp of all pixels, which allows us to use interpolation on the pixels in order to find the average color for that pixel in our new mid-way face image. Combining both processes together, we get the following midway image:

The Morph Sequence

In order to produce the morph sequence, I utilized the methods that I implemented above for affine transformations and inverse warping for the two images and their mean shape, but I also added warp_frac and dissolve_frac as parameters to the functions implemented. For each of the 50 frames, I incremented the two parameters (in between 0 and 1), using them to interpolate both the point and color values-- allowing us to smoothly transition between the appearance/shape transformations of the original images.

The "Mean Face" of a Population

In this part of the project, I used the Danes faces dataset in order to compute the "mean face" of the dataset's population. In order to do this, I computed the average face shape of the full frontal face, neutral expression, diffuse light subpopulation (Image Type 2), warped each of the faces in that subpopulation to that average shape, and finally computed the average of the warped faces.

As we can see above, generally the original images of the people are warped such that they take on certain physical characteristics of the mean face, such as its face/eye symmetry, thinner nose bridge, and smaller overall face-- among other features. We can use this warp operation with the mean face on our image of Robert. I then warped Robert to the Mean Face's geometry and vice versa. I used photoshop to have both of the image's dimensions and general alignment/positioning be the same. As you can see below, Robert's warped geomtetry now exhibits some of the similar features that we saw above: smaller face, more symmetrical face, and thinner nose bridge. His eyebrows do look off as transforming his previous bushier, assymetrical eyebrows into the thin, symmetrical ones of the mean image is quite tough without precise keypoints.

Caricatures: Extrapolating from the Mean

In order to create a caricature of Robert we had to first utilize the subpopulation mean face that we computed before and calculate the extrapolated points with the equation (1-alpha)*P + alpha*Q instead of our normal 0.5*P + 0.5Q average, where P and Q are our two point sets for the two images and alpha is a value denoting the "intensity" of our caricature.

Bells and Whistles: Change Ethnicity of George Clooney to South African:

Finally, I decided to "change" George's ethnicity to South African by using the average South African male face. In order to do this, I redefined the corresponding keypoints between the two images and computed the midway of the two images (a similar process to the one above for Mid-Way), which shows both the morphing of the shape and appearance. I also warped George's face to the Average South African face's geometry and vice versa to show the morphing of shape only. Additionally, I did a simple cross disolve between the two original images to show the morphing of only appearance.