Overview: In this assignment, we perform various transformation based morphs based on different images and image populations. Corresponding points are defined in the same key places on multiple images, and a triangulation of these points is created to partition the images into corresponding triangle meshes. Affine transformations can then be calculated for each pair of triangles to warp the pixels of one image into the other. We can use this technique to produce morph animations, population face means, and caricatures based on extrapolation.

Defining Correspondences

First, we want to define pairs of corresponding points on a picture of me and a picture of George Clooney, with which we will later produce a morph animation. I used ginput to choose points in the same order for both images. I then obtain a midway shape of these two sets of points by averaging them. I use Delaunay triangulation on this midway shape to obtain the triangulation which I will use for morphing. Below shows the two images and the points over them along with the calculated triangulation.

Computing the "Mid-way Face"

We can compute this by warping both of the above faces into the average (midway) shape and then averaging them together. To warp a face into another shape, we compute an affine warp between every corresponding pair of triangles between the two images, then apply the inverse of this affine warp to the triangle in the new shape to get all corresponding pixels from the original image (inverse warp). The affine transformation T is computed from the following equation: Tx = y, where x is the 3x3 homogeneous coordinate matrix of the triangle from which we want to transform, and y of the triangle which we want to warp into. We get T = y * x^-1.

Below shows the the mid-way face between the two original images.

The Morph Sequence

The morph sequence is similar to computing the mid-way face, but now we iterate over fractions determining what shape and color cross-dissolve should be used (the fraction was 0.5 for the mid-way face). The fraction starts from 0 (shape and color of my face) and increments over 45 frames to 1 (shape and color of George Clooney's face). Chaining these images together into an animation gives a smooth morph sequence which is shown below. The gif "bounces" back into my face at the end before restarting to give a smoother effect.

The "Mean Face" of a Population

I computed the mean face of a small population of faces based on annotated data provided by the IMM Face Database. This involves first morphing every face in the dataset into the average shape of all face shapes in the dataset. Below is a few examples of this morphing (before and after).

With these faces morphed into the average shape, we simply average all these images to get the average face of the population in the dataset. With the same procedure used for morphing faces above, we morph my face into the average face shape of this population, and also morph the average face into my face shape.

Average shape face

My face warped into average shape

Average shape face warped into my face shape

Caricatures: Extrapolating From the Mean

We can extrapolate from the mean to create caricatures of my face using this equation:

C = (1 - alpha) * my_shape + alpha * average_shape. The larger alpha is, the closer it gets to the dataset average shape, and alpha > 1 means that it is extrapolating from this dataset average shape, creating a caricature with emphasized features from the average face. alpha < 0 means that it is extrapolating in the opposite direction, reducing emphasis on features from the average face while emphasizing features in the original face. Below shows various alphas (note that alpha = 0 means the original face shape is retained).

alpha = -2

alpha = -1.5

alpha = -1

alpha = -0.5

alpha = 0

alpha = 0.5

alpha = 1

alpha = 1.5

alpha = 2

Bells and Whistles: Class Students Music Video

Below is my part of the morph chain of students in the class, as well as the music video.