Alex Jiang

CS 194-26: Project 4

Face Morphing

Overview

        The face morphing algorithm is one of the most iconic projects in this class, and  utilizes many different concepts learned so far. Alongside its accompanying exercises, the face morphing further tests the ability to create interactions between images, much like blending in the previous project. As one of the first projects introduced to me at the start of the semester, I was looking forward to implementing it and I had a lot of fun with the results :)

1. The Face Morphing Sequence

I implemented the core of this project, the face morphing procedure, much like how it was outlined in the spec. First, the user selects points on each image that match each other (as well as the corners of each frame as anchors), which are stored in arrays. We triangulate each image according to these points using Delaunay Triangulation. We then interpolate the images according to the points as well, before progressively warping them to match the opposing image's point maps. Finally, we weight each image to create the actual morphing effect, where one starts at 0% presence while the other starts at 100% presence. They gradually trade off their dominance, blending together and creating the final product.

Here, you can see the result of my face morphing algorithm on myself and Hollywood actress Anna Kendrick.

The GIF above is looped, but there are 45 frames to transform from one base image to the other. All of the individual frames are displayed below, with the exact Midway frame (frame 22) labeled accordingly:

Due to the extreme amount of differences between my own face and Anna's (essentially opposites in every feature and characteristic), the Midway itself doesn't turn out spectacularly, but it still blends well, the overall morphing sequence is quite smooth, and I'm proud of the result.

2. The “Mean Face” of a Population

Next, I worked with the Danes face database to demonstrate how multiple faces can be blended together, rather than just two. Using the given annotations, I created a single average point map to represent the total image population. Then, I warped each subject into that average point map, taking each pixel array result and averaging that as well. Finally, we create the representative image as follows:

We then morph myself into the average directly using the point maps, and vice versa:

The transformation doesn’t work great here, likely because of both the angle of my head and also the vastly different hairstyle. Even the facial features are pretty drastically different. Still, you can sort of see the shapes blend in the proper fashion and the main focal points of the image (eyes, dose, mouth) syncing up to a certain degree. For more context, here are a couple more examples of Danish images warped towards the average:

        These ones generally seem much less dramatic, which makes sense since the pictures are taken in similar fashions and the average also takes the originals into account upon creation.

3. Caricatures

        Here, we take the difference we already established between myself and the average (which was large) and extrapolate based on that. This was calculated by simply comparing the differences between each corresponding point (e.g. point 1 of Alex’s point map and point 1 of the average’s point map), and then multiplying that difference before adding it to the original image. Since my face can only scrunch so much while looking tolerable at all, I threw the caricature in the other direction, as you can see below:

What’s most interesting about this one to me was that my face starts angling downwards, which makes sense since my chin is lifted higher than the average’s chin. Thus, when it’s reversed, my head starts tilting the other way, which creates an interesting effect.

4. Bells and Whistles

        Since I already saw a massive transition from my face to Anna’s, which changed gender, ethnicity, age, and expression (so, basically everything), I decided to pursue this a little further and look at the results from in a more general sense. First, to more directly build upon my first face morph, I tried using a general female facial structure:

        At a first glance this looks pretty bad, but the more I look at it, the less bad it feels to me. The smaller face actually does seem reminiscent of a young Asian female, and if I smudge the image a bit in my head I could definitely come up with a couple girls that look like that. The hair also turned pretty feminine, so overall it was a surprisingly effective transformation, if you can see past the horrifying jagged lines.

        Next, I tried modifying a more specific aspect of the image: the smile. I did this in an attempt to remove the smile by mapping the points to a more neutral expression:

This one was… close. Not really there, mostly since you can still see the teeth, which is slightly creepy and also makes the frown much, much less convincing. The dimples also didn’t drop nearly enough since the point map comparison wasn’t dramatic of a change besides in the mouth area, so the rest of the face isn’t conducive to a bored nor downtrodden expression.

Moving on, I thought that some of the previous expressions, so I combined this facial-adjustment concept with the caricatures from before. Essentially, I took the base image and did the same style of smile adjustment, except I magnified the point map with a completely over-the-top magnitude. In other words, I made some images that, while not very convincing in terms of realism, came out as very amusing results.