CS 194-26 Project 3

Jaiveer Singh

Computing the "Mid-way Face"

The mid-way face is a face that exists halfway between two subjects' faces, both in terms of facial structure and in terms of color values.

For this series of experiments, I have combined the faces of Barack Obama and Joe Biden.

Input Image 1 Input Image 2

First, the faces were labelled with corresponding keypoints, as shown below.

Labelled Image 1 Labelled Image 2

The spatial average of the corresponding feature points were used to produce the average triangulation, so as to minimize the likelihood of malformed triangles in either extreme of Obama or Biden raw images. The mean face, in isolation and with annotations, is shown below.

Mean Obama-Biden Face Labelled Mean Obama-Biden Face

The Morph Sequence

Extending the technique in the previous step and interpolating across the range of 0 to 1 (ie, all-Biden or all-Obama) produces an elegant face morph gif shown below:

The "Mean Face" of a Population

These techniques are not limited to only pairs of images. Given a large dataset of faces, we can conduct analysis on what the average face looks like. This next series of steps relies on a Brazilian faces dataset, found here.

First, we will compute the average face out of the 200 non-smiling faces present in that dataset.

We can morph several individual faces from the dataset to the mean face, as shown below:

Input Face Morphed Face Labelled Morphed Face

We can also morph Obama's face into this same mean face format, and additionally morph the mean face into Obama's face format:

Obama Mean Obama-to-Mean
Mean Obama Mean-to-Obama

Caricatures: Extrapolating from the Mean

Now that the average face is known, instead of morphing an input face to appear closer to the average face, we can instead exaggerate the differences between the input face and that average face to produce a caricature.

The resulting caricatures with increasingly large alphas (ie, more exaggerated features) are shown below:

Alpha=1 (unchanged) Alpha=1.25 Alpha=1.5
Alpha=2 (features doubled) Alpha=3 Alpha=5

As is clearly exemplified by these images, increasing alpha to increasingly high number makes Obama's face wider and eyebrows more pronounced. Of course, increasingly higher values of alpha lead to a breakdown in regular face geometry due to the distortion introduced by extrapolation.

Bells and Whistles: dlib

One of the most tedious parts of this entire process is the manual selection of features on each image. This is error-prone and hardly repeatable; the same human annotator is unlikely to precisely label the exact same points in the exact same locations each time.

One solution to this variability is to employ the use of dlib for an out-of-the-box solution to label key feature points in faces. Extracting feature points on each of the input Obama and Biden faces and producing a morph gif in the same way leads to a noticeably smoother result:

Labelled Image 1 dlib-Labelled Image 1
Labelled Image 2 dlib-Labelled Image 2
Morph dlib Morph

Bells and Whistles: Presidential Procession

Thanks to the auto-labelling capabilities of dlib, it is now possible to generate a long-form gif combining the faces of the most recent US Presidents (those with color photographs) in order! Enjoy the positively patriotic presidential procession!