Project 3 CS 194-26: Face Morphing!

By Diego Uribe

Project Description

In this project I worked on producing a morph animation of my face and George Clooney's face. I also computed the mean face of the Danes population of faces. Finally, I extrapolated from the mean face of the Danes population to create a caricature of my own face!

Part 1: Defining Correspondences

For this part I first selected two pictures that I wanted to perform a morph on. First, I selected a picture of George Clooney. Then I took an image of my face such that my face was about where George's face was in the picture. I then resized my image so that it had the same size as George's. After doing this, I defined pairs of corresponding points on the two images using cpselect. For this I focussed on mapping eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc. I saved the corresponding points on a CSV file so that I do not need to recompute them! Finally, I used the points selected from my face to compute a delaunay trangulation which will be used for morphing in the next section.

Part 2: Computing the "Mid-Way Face"

In this part I computed the mid-way face of my image and George's image. For this, I wrote an algorithm that did the following:

1. Iterated through the triangles in the delaunay triangulation of my face.
2. For each triangle, it computed the average traingle of the triangle location in my face and the triangle location in George's phase.
3. Then, it computed an affine transformation matrix from the average triangle to my face's triangle and from the average traingle to George's triangle.
4. THen, it computed the pixel coordinates inside the average trinagle and transformed them into my face's triangle and George's triangle using the affine transformation matrices.
5. Finally, it determined the RGB color of the pixel coordinates in my face and George's face using interpolation and assigned this values to the corresponding pixel coordinates in the mid-way face.

Here are the results:

Part 3: The morph sequence

In this part, I extended the algorithm described above to compute a full morph sequence between Diego's and George's faces. This morph sequence consisted of 45 images, where image 1 is Diego's face and image 45 is George's face. The way I extended the algorithm above is that instead of calculating the average shape of the triangles between Diego's and George's face, I used a warp_frac and a dissolve_frac to control how much influence wach image has on the "mid-way/morphed" image in terms of the shape and the colors.

Here are the results:

Part 4: The "Mean-Face" of a Population

For this part, I worked on calculating the mean face of the Danes dataset. I computed the average face shape of the entire population for image type 1 which is the full frontal face with a natural expression image. The algorithm consisted on the following process:

Extract the images and their respective corresponding points from the dataset. Compute a delaunay triangulation for one of the images. I did it for the first one. I used the given set of points to compute this and added 4 more points (corners of the image). Then I computed the mean_shape of all the faces in the danes dataset. This involved averaging the corresponding points of each of the images. Then I warped each of the faces in the dataset into the average shape. Finally, I computed the average of the warped faces to get the mean face of the population

Here are the results:

My face warped into the Dane's Mean Face average geometry

For this part, I used another image of myself. This was taken horizontally to better match the Danes Mean Face dimensions. I also had to recompute the pairs of corresponding points between the two images. Then I just proceeded to morph the two images using the set of corresponding points.

Dane's Mean Face Warped into my face's geometry

Part 5: Caricatures: Extrapolating from the Mean

For this part, I used the Dane's Mean Face and my face (horizontal orientation). The first step in the algorithm was to subtract the mean population face from the face, call this difference delta (see image below). Delta represents the defining features of my face. Then a caricature consists of my original face plus alpha * delta, where alpha is a constant between 0 - 1. Below I show the results for various values of alpha.

Part 6: Bells and Whistles

I change the gender of my face into the average women face of the danes dataset. For this, I first calculated the mean face of all danes women. Then I calculated a morph sequence between my face and the women face.