Overview

In this project, I was able to take what I learned from lectures and implement those concepts into actual code by morphing faces! First starting out with finding the midway point between two faces, I was able to eventually make a relatively seamless transition between one face to another by morphing between frames of the mid-images. I also analyzed the average face of the FEI Face Database and used it as a reference point to perform cool transformations, such as warping to the average face and caricatures!

Part I: Defining Correspondences

To begin the project, I first got a picture of myself and a friend, making sure that they were the same aspect ratio, and then decided that I would work on morphing my face into hers. From there, I defined a set of 53 coorespondence points using ginput that mapped to various features on our faces, while also including the four corners of the image, before then computing the Delaunay Triangulation of the average of each of those corresponding points. The idea behind using the midway shape as the triangulation is to lessen any possible triangle deformations that may occur during the morph. The original images and their triangulations can be seen below:

Part II: Computing the "Mid-way Face"

Once I had decided on the correspondence points and triangulation that I wanted to use, I needed to use those to compute the "Mid-way Face", or halfway blend between the two faces. To do this, the first thing I did was create a function that could find the Affine Transformation matrix A between two sets of points using least squares. Then, for every triangle in the triangulation, I computed the affine transformation from the coordinates in the original image to the mid-image, applied it over all pixels within the triangle, and then put the color at that coordinate into the mid-image. Lastly, I combined everything at the end by averaging the colors, to get the resulting "hybrid" face. Note that the previous too images A and B were shown earlier right above:

Some issues I had with this part were how to appropriately process the pixels in parallel instead of iterating through them. I eventually found a way to simply run the Affine Transformation on the entire set of pixel coordinate values at the same time just by using numpy arrays. Additionally, I used Inverse Warping instead of Forward Warping to ensure that the pixels in my images were guaranteed to have a correct unique mapping to the mid-image. One slightly more irritating issue I had was that I mixed up the row and column coordinates in my Affine transformation, and it caused some very strange artifacts to appear. This was more of a cause of my unfamiliarity with numpy, but it was still one of my most frustrating bugs to fix.

Part III: The Morph Sequence

Once the generic algorithm for computing a mid-image was completed, the ability to compute the whole morph sequence followed quite naturally. I defined a function that warped the two images to different mid-way images in between, at different warp fraction values and dissolve fraction values. Essentially, for a 45 frame sequence, I iterated from 0 to 1 with a step of 1/45, using that value as the t value to determine the weight of each image when creating the average shape at that timestep. I also used that same value to determine how to cross-dissolve the images together in the mid-image. The result of this is a short GIF that demonstrates the morphing sequence!

Part IV: The "Mean Face" of a Population

I moved on to analyzing the different faces within datasets; in particular, I chose to use the set of spatially normalized, neutral expression faces from the FEI Face Database as my source. First, to compute the average shape across the entire population, what I did was take all of the sets of coorespondence points from each image and average them together, getting an average shape. Then I proceeded to try warping several of the faces from the population to the average. Below are some results of that, including the computed average face and the full morph between the two:

Additionally, here are the results of me being warped to the geometry of the average face, and the results of the average face being warped to me. Due to the way the image is cropped and the way the dataset previously computed its correspondence points, the results are very amusing to see. I really do have a big forehead:

Part V: Caricatures: Extrapolating from the Mean

By extrapolating from the mean, I managed to produce different caricatures, with a further exaggeration at higher factors! The first was created using a warp fraction of 2, while the second was created using a warp fraction of 3. By really emphasizing the difference of features, the caricture becomes more and more outrageous looking:

Part VI: Bells and Whistles

I teamed up with a group of classmates to create a Youtube music video of our face morphs! I was responsible for warping someone else's face to my own, and the gif for that is also below:

Funnily enough, although the curly hair and glasses left some artifacts as the morph occurred, the end result was still really great to look at!

This project for me was actually fairly challenging, just because I spend a lot of time debugging implementation issues in numpy. Overall though, the experience as great, and I really feel that I learned a lot about the techniques behind face warping. As a bonus side note, I've also attached a result of one of my warpings gone wrong :)