CS194-26 Project 5: Face Morphing

Alexander Chu, ID#23460953

Morphing

Defining Correspondences

To perform face morphing, I implemented a simple user interface to collect points on the user's face. This would collect points in order. The expectation is that corresponding points are collected in the same order in the other image. Below, we show the point collection interface and the delaunay triangulation of those points.

The Mid-Way face and the Morphing Sequence

I computed the morphing seuqence of images by taking the weighted average of the control points on both images, and computing its delaunay triangulation. I then used tsearch on this triangulation to compute the pixel from each image to use for a given pixel on the output image, with the position under an affine transform determined by the coordinates of the triangle vertices. The pixels are blended using a weighted average specified by a dissolve factor. By iterating accross the range of dissolve and warp factors from 0 to 1, I generate a morphing animation. To generate the mid-way faces, I compute the morph element with the dissolve and warp factors equal to 0.5.

Image A Image B Mid-Way Morphing Animation

The Mean Face of a Population

Danes Database

I computed the average of the Danes dataset from the website. I scraped the annotation points of each face from the asf file, and matched each image with its annotation points. To get the geometry of the average face, I computed the average positions of all of the annotation points, and computed its delaunay triangulation. I then used this triangulation to map points on the average face to points on each of the images. I morphed each image to the average face geometry according to the triangulation, and averaged the aligned faces. Results are shown below. A subset of the danes morphed to the average geometry is shown below.

Source Image
Image Morphed to Average Geometry

Average Face:
My face warped into the average face The average face warped into my face

When my face is warped to the average Dane face, or when the average Dane face is warped to my face, the result is heavily distoted. This is because I don't have the Danish facial structure. Since I'm not similar to the sample population, both results are very distorted.

CS194 Database


After the CS194 face datbase got enough submissions, I also computed the average of those faces. The resulting images are shown below:


Source Image
Image Morphed to Average Geometry

Average Face:
My face warped into the average face The average face warped into my face

This result is somewhat better. Since this average face has a similar facial structure to me, the two warped images are not as distorted as the previous pair with the Danes. Also, the sample population is relatively small, and contains me, so the resulting average is likely to resmble me more.

Caricatures: Extrapolating from the Mean

I used the average from the CS194 database to compute a caricature of my face. To do this, I morph my face to the average face, the average face from my face, and add some multiple of the residual to my face. A multiple of 0.25 seemed to work best. As shown in the caricature below, it appears that my most distinguishing features are my nose and the dark circles around my eyes. Maybe I should get more sleep.

OriginalCaricature

Bells and Whistles

Changing Gender

I used the image of the average Han Chinese woman, and an image of my own face of similar dimensions, and computed how I would look as a woman. I did this by taking the difference between my face and that of the average woman, and subtracted some multiple (0.25 worked best) of the difference from my face. The result is as follows:

Average Chinese Woman My Face Morphed Result

The morphed results appears slightly more feminine than my original face. It is also brighter, since the average woman's face has a lighter skin tone than me. The morphing also seems to have changed my expression from a frown to a smile, since the average image seems to be of smiling faces. Some artifacts are present on the right and left side of my neck. These are points outside of the convex hull of control points that I defined, so the only triangulation data for these points would be between the face points and the default points I added in the corners. Since these triangles are quite narrow, triangular artifacts are somewhat visible.

Morphing Music Video: Squirtle Evolution

I used the face morphing algorithm to compute a video of the Pokemon Squirtle morphing into a Wartortle and then a Blastoise. This is different from faces, since there are not necessarily clear correspondences between the parts of Pokemon. I tried concentrating the control points on the obvious features of the Pokemon, like the arms, legs, shell, and facial features. Some of these features do not hav any clear correspondences, however, like Blastoise's cannon, which appears out of nowhere in the video. The background theme is a remix of the Pokemon evolution song by TheAppleFreak on newgrounds.

CS194 Face Database

The face databases linked to us from the class were really cool! Feeling inspired, I wrote an app that would allow us CS194 students to build our own face database. This was originally going to be an crowdsourced face chaining app, where my server would compute the face chain of all submitted images in the background as they were uploaded, and display the video on the page. However, I enountered some difficulties with ffmpeg on my server. Also, this automated approach would have no defense against trolls who would upload non-faces or faces with poorly done control points to the service (there are a couple of these in the database, so caveat emptor!).

Face Chain

I participated in the face chain on Piazza. Here's a gif of Rachel Albert's face morphing into mine:

What I learned

The coolest thing that I learned while doing this project is how versitile triangulations can be for interpolating the contents of one image from another. I was suprised at how smooth the resulting averages were, given that the tsearch function seemed to map points discontinuously from one space to the triangles of another.