CS 194-26 Image Manipulation and Computational Photography

Defining Correspondences

The goal of this part is to annotate points in both of the images in order to have correspondances when performing image morphing. For this part, I’ll be using images of George Clooney, image A, and Barack Obama, image B. You can see the original images below.

Images of Barack Obama and George Clooney

For image annotations, I used total 81 points, including the four corners of the images. 44 of those images is highlighting the shape of the faces, 10 points are for eyes and eyebrows, 7 for nose, 5 for mouth, 6 for chin and 5 for neck and tie. After finding the correspondences, we find the average shape, which is the average of corresponding points, then construct the Delaunay triangulation out of this points. You can see examples triangulations below.

Delaunay triangulations

Computing the "Mid-way Face"

We first start computing the mid-way face of our images A and B. For this, we compute the average shape, which is the average of the correspondence points, warp both faces into that shape and then, finally, averaging colors together. In order to do this, we perform affine transformation from each triangle in image A to the triangle in the mid-way face image. To find the affine transformation matrix we use a simple system of linear equations. Also, we perform inverse interpolation. You can see the mid-way face below.

Original, The Mid-Way Face, Original

The Morph Sequence

For this section I used 45 frames creating the morphing sequence. Here are some images from the sequence.

Morph sequence

And here is the animation.

George to Barack Animation

The "Mean face" of a population

For this part, we want to find the average face of the population. However, simply averaging all the images together would not work and would be blurry, as the faces in Danes dataset are not aligned. The dataset comes with manual annotations of the images, which helps as to perform morphing in order to get the average face. First, we find the average shape, which is the average of all the shapes, or the average of all the annotation points. Then we morph each image in the dataset to the average shape. You can see some images below.

Original and the morph to the average shape

Original and the morph to the average shape

Original and the morph to the average shape

After doing the two steps mentioned above, we go ahead and cross-dissolve all the images together to find the average face. You can see the result below.

The Average Face

After finding the average face, I morph my image to the average face shape and the average face to the shape of my face. You can see some distortions in the morph of my image to the average shape, which is probably because of correspondence points and my selfie. As you can see on the average images it captured some of my facial features, like the nose and chin.

My image, Morph of my image to average shape, vice versa

Caricatures

In this step, we extrapolate from the population mean we calculated and morph my image to get those distinctive features. You can see some examples below.

Caricatures, t = -0.25, -0.5, -0.75, -1