CS194-26 Project 4: Face Morphing and Modelling a Photo Collection
Joey Barreto
Face Morphing
To morph two images into each other, one cannot just average them even if the images are aligned similarly because individual features within the images may not align. For example, with two faces, the heads may be centered and proportioned similarly but one person may have a higher nose, the other more widely-spaced eyes, etc. To map features to features, one must first manually select corresponding feature points (e.g. the first points of both images' feature points mark the left eye pupil). If you want to warp the images into a halfway morph between the two, first you would average the feature point positions and compute a Delaunay triangulation on those points. The corners of each triangle are mapped to their respective feature points in the two images, establishing corresponding triangle regions in the two source images. An affine transformation is computed from each triangle in the halfway geometry to each respective triangle in both of the source geometries. These affine maps allow us to determine which pixels in the source images contribute to which pixels in the morphed image, to allow color transfer that complies with the triangulation. To get a morph sequence, you would do this for a number of different choices of a parameter, let's call it t, that controls the averaging of the source geometries (the midway frame of each morph sequence corresponds to t=0.5).

In the example below, the source images are pictures of Hillary and Trump, and one of the intermediate triangulations is displayed on top of its corresponding frame.

Hillary

Donald

The midway point

The full morph

My face

My roomate's face

The midway face of our faces

The full morph between my and my roomate's faces

The Danes Dataset
For the below images, an feature point-annotated dataset of Danish male and female front portraits is used. The average face of a population can be found by averaging each facial geometry, warping each face to that geometry, and then averaging all the warped faces. In the faces below, facial distortion corresponds to deviation from the mean facial features of the population (male or female, in this case).

The average male face

A male face morphed into the average male geometry

Another male face morphed into the average male geometry

The average female face

A female face morphed into the average female geometry

Another female face morphed into the average female geometry

The following two tables are exactly the same as above, but using smiling versions of all the same images.

The average male face

A male face morphed into the average male geometry

Another male face morphed into the average male geometry

The average female face

A female face morphed into the average female geometry

Another female face morphed into the average female geometry

I warped my face into the average male Danish geometry, and also warped the same average face into my facial geometry.

The average male face

My face morphed into the average male geometry

The average male face warped into my geometry

Caricatures
Here I create a caricature of one of the male Danish faces by amplifying how his facial geometry and appearance differs from the mean. I warped his face to a new geometry = his geometry + a * (mean geometry - his geometry). Next I warped the mean face to this new caricature geometry to get the alignment right, and then I made the appearance of the caricature = caric app + a *(warped mean app - caric app). The parameter a lies between 0 and 1 and controls the caricature effect. I also made a caricature of my own face with reference to the same dataset.

Male face warped to male face + 0.6*(mean - male face) geometry

The mean face warped onto male caricature geometry

Male caricature geometry with male face + 0.6*(mean - male face) appearance

My face warped to my face + 0.4*(mean - my face) geometry

The mean face warped onto my caricature geometry

My caricature geometry with my face + 0.4*(mean - my face) appearance

We can use the features of specific subpopulations to make a face look happier, older, more male/female, etc. Below, I make one of the female faces look more male by morping her face into the average male geometry and then averaging the morphed image and the average male image.

The 14th female face

The average male face

The female face morphed into the average male geometry

The appearance (colors) of the female face averaged with the average male face

The morphed female face averaged with the average male face