CS 194-26: Image Manipulation and Computational Photography

Project 4: Face Morphing

Tony Pan, CS194-26-adn



Overview

This project involved triangulations for two images, and using affine transformations to make "midway" blended pictures as well as morph from one image to another.

Part I: Defining Correspondences

For the faces to be morphed correctly, it's important to define some correspondences first. These corresponding pairs of points form a consistent labeling on each face, so that a relatively seamless blending can be achieved. A few example correspondences are shown below.

Correspondences for me, my roommate, and George Clooney.

Given these points, it's relatively simple to create a triangluation to use in morphing. I chose to use scipy to compute a Delaunay triangulation on the midway shape (mean of the two point sets). This yielded a triangulation that generally works well for most morphs. The example triangulations below were computed using midway shapes, with the triangles being mapped to each face's corresponding point set.

Delaunay triangulations for me, my roommate, and George Clooney.

Part II: Midway Face

The midway face is computing by finding the affine transformation from each triangle in the original face images to its corresponding position in the midway triangulation. After being warped, the corresponding triangles from both face images are cross-dissolved by 50%. This creates an image that with shape and coloration that's blended midway between both faces.


Part III: Morphs

Having calculated the midway face, it's relatively easy to create a morph. Instead of warping geometry and blending color by 50%, we can create a series of images warped and blended with values increasing from 0 to 1. In this case N=45 was used, so warp_frac and dissolve_frac values of [0, 1] were used with increments of 1/45. I chose to morph from face A to B and back to A, so I just repeated the process, but with A and B switched. Playing through each 'frame' in sequence yields a gif of face A morphing into B, and then back into A.


Part IV: Mean Face

For this part of the project, I used the Danes dataset to calculate the mean set of correspondences for each face, as well as the mean face of the entire population. Because the dataset consists mostly of male faces, the average is a reasonably "generic" male face. The mean face is below, as well as some examples of individual faces morphed into the average geometry.


These are some example images of individual faces morphed into the average geometry. The original images are on the left, and the morphed images are on the right. The first image was largely unchanged, except for a slightly wider jaw. The second and third images exhibit more visible differences in position of facial features, as well as relative face shape.


The images below are of my face morphed into an average geometry and an "average" face morphed into my geometry. The average geometry and face were computed from the Danes dataset. There are some pretty visible changes in my morphed face in the shape and positioning of various features (eyes, nose, jaw). Similarly, the average face becomes wider with a sharper jaw when morphed into my geometry.


Part V: Caricatures

As can be seen above, my caricature(right) is essentially an exaggeration of the differences between my face(left) and the average face in the Danes set. Because my face is somewhat wider with a sharper chin, these features are amplified in the caricature. Similarly, my eyes are somewhat further apart than the average face, so in the caricature this distance is increased.


Bells & Whistles

Change Gender/Ethnicity

For this portion of the project, I took a photo of my roommate, Mitchell, and morphed his face's shape and appearance into that of an average Danish woman from the Danes dataset (Mitchell and the average Danish woman shown above). From left to right: Mitchell, Mitchell's shape morphed, Mitchell's appearance morphed, both shape and appearance morphed


All The Presidents

A music video without the music. I originally wanted to make a video/morph of all the presidents in sequential order, with the Animaniacs Presidents song as background music. Time restrictions kept me from adding the music, but the video is complete in full, with each president of the US appearing in order of first term served. (Re-elected presidents and consecutive terms aren't counted, just distinct presidents).