Face Morphing

By Joshua Levine

Defining Correspondences

On each image, I selected 49 corresponding points by hand, and also programmatically added the corners as points.

I then computed a Delaunay triangulation on the mean shape.

Computing the "Mid-way Face"

I start by computing the mean shape and then computing a Delaunay triangulation from that. I then iterate over timesteps t which is between 0 and 1, inclusive, and computed a partial warp using warp_frac=t.

For each time step, I iterate through each triangle and compute the inverse affine transformation matrix that I use to transform to points in the original image. I then use a RectBivariateSpline to interpolate pixels in the original image. After the shape for that timestep has been computed, I use dissolve_frac to compute a weighted average of the pixel values to get an average color in addition to the shape.

I struggled a lot with indexing in this part. It was especially hard to account for the fact that arrays are indexed differently from the images themselves, so in code, they are challenging to deal with intuitively. Below is the triangulation on the two images.

img img

Below are the original images, followed by the midway face

img img img

The Morph Sequence

In this part, I iteratively called the morph function that I used in the previous part and created a gif from the resulting images. The gif is below

img

The "Mean Face" of the Population

For this part, I used images from

M. B. Stegmann, B. K. Ersbøll, and R. Larsen. FAME – a flexible appearance modelling environment. IEEE Trans. on Medical Imaging, 22(10):1319–1331, 2003.

I started by converting the points in each .asf file to arrays and saving them as .txt files so they are easier to use later. I also add the corners as corresponding points to each image. Then, I compute the mean shape, morph each image to that shape, and compute the average pixel values from the morphed images. I took the mean of all full frontal images without facial expressions.

Here is the mean face

img

Here are some of the images morphed into the mean shape:

img img img img

Here is my face warped into the mean geometry.

img

Caricatures: Extrapolating from the mean

For this part, I used the men in the database that I used in the previous part. I then computed the mean shape and found the difference between my face shape and the mean. Then, I added a 1.7 times that difference to the mean shape to get a caricature. See the result below.

img

Bells and Whistles

For this part, I found computed the mean male smiling face from the population used in the previous parts, and morphed my face to that image to change my expression. See the mean male smiling face, and mine morphed into it below. The last three images below show morphing just the shape, just the appearance, and both.

img img img img

In addition to morphing my expression, I also tried to morph my face with the mean face of from the pictures of men that were rotated 30 degrees. As you can see below, the result was very peculiar, but interesting nonetheless. I displayed mean image as well. The reason it does not work well is because I cannot correctly mark pairs of corresponding points on the rotated image. The last three images below show morphing just the shape, just the appearance, and both.

img img img img