Project 4

Face Morphing

Defining Correspondences

Because cross dissolving doesn't necessarily create a smooth transition between two images, we need to consider not only appearance but also shape in morphing.

First, I defined pairs or corresponding points on two images I was interested in morphing. I did this by hand using ginput and plot commands in python. Between the two images, the points must be selected consistently, so if point 1 in image A is under someone’s left ear, then point 1 in image B must be a point under the other person’s left ear. I store these points so that I don’t need to click 50 points on each image every time I run the code.

Triangulation produced for Clinton (top) and Obama (bottom)

Original Clinton Image

Original Obama Image

   

Computing the “Mid-Way Face”

I compute an average point between every pair of points to define the “average shape” between image A and image B. The high level idea is that if you morph A into the average shape and the average shape into B (while also morphing the colors), you can get a smooth transition from A to B.

On the average shape points, I compute a Dalanauy triangulation that defines a triangulation that doesn’t have overly skinny triangles. To morph image A into the average shape points, I use the triangulation I found and define the triangles in image A as they are defined by the average triangulation (i.e. if the first triangle in the Delanauy has points 1, 4, and 6, then the triangulation applied to image A would be between image A’s points 1, 4, and 6). The same process is repeated with image B.

For the corresponding triangles between the average shape and image A, I compute the affine transformation matrix that warps the average shape’s triangles to image A’s triangles for every triangle. I compute these for the triangles in B as well.

Then, I loop through every triangle in the average shape. For every triangle, I use the draw_polygon function mentioned in the spec to return all of the pixel locations within the triangle. Then, I look up the transformation matrix from the average shape to image A for that triangle and apply it to all the points returned by draw_polygon, and I sample the pixel value from the corresponding location in image A. This produces image A warped into the average shape. The same can be done for a second image B.

Clinton Morphed into Average Shape

Obama Morphed into Average Shape

 

The Morph Sequence

To produce a smooth warp between image A and B, use warp_frac and dissolve_frac to control the amount of shape morphing and color morphing, respectively. In the range[0,1], I incrementally increase warp_frac and dissolve_frac such that frame 1 (where warp_frac and dissolve_frac = 0) looks like image A, and frame 45 (where warp_frac and dissolve_frac =1) looks like image B. At each step, I crease warp_frac and dissolve_frac by 1/45 because I want 45 frames total.

I compute 45 images that show the gradual morph between image A and B. This means that at each step, the average shape is defined by (1-warp_frac)*image A points + (warp_frac)*image B points and then I calculate the warping from image A to this average shape and image B to this average shape. At the end of the computation, I cross dissolve them using the dissolve fraction (similar to the way the warp_frac is used). At the next step, I increased warp_frac and dissolve_frac by 1/45 because I wanted 45 frames. Finally, I use photoshop to string together the 45 frames into a gif.

Mid morph between Clinton and Obama

Hilbama!

The “Mean Face” of a Population

I used the Danes dataset of annotated faces for this section. First I computed the average shape of the entire population by averaging the corresponding annotated points in each image. Each image’s annotated points are given a weight of 1/40 (since there are 40 images) and added together to compute the average.

I warped all of the faces in the dataset into the average points using the same method described above (triangulation + affine transformation). Here are a couple examples from the dataset:

Average Dane

First Dane in dataset

Second Dane in dataset

First Dane morphed into average Dane shape

Second Dane morphed into average Dane shape

Note that for this dataset, the females are labeled with an ‘f’ in the filename. I changed these to ‘m’ for convenience sake so that they match the other files.

 

Caricatures: Extrapolating from the Mean

Because faces span a subspace, if we extrapolate faces in specific directions within the subspace, we can create caricatures. In order to compute the caricature shape, I add the image of interest (my face) with a scalar multiple of difference between that image and the average image (Danish average face). If the scalar is positive, the Danish features are exaggerated, and if the scalar is negative, the differences between my face and the Danish average is accentuated.

Average Dane

My normal face

More Danish, a=0.8

Less Danish, a = -0.8

Bells and Whistles

I decided to morph my face into the average Chinese male using the same methods as above.

My face

Average Chinese man

Color and Shape Morph

50% color morph

50% shape morph