Project 3: Face Morphing

Computing the Midway Face

To compute the midway face, the first step is to define corresponding points for the paired images. Below is a sample of two images (me on the left Steven R McQueen on the right) with corresponding points.

The next step is to compute a triangulation of these points, for this I used the scipy.spatial.Delaunay triangulation method. Below is a visualization of the triangulated faces.

Finally, to compute the mean face, I perform the following steps

1. Compute the mean set of points from the two sets of points based in the two images.
2. Apply the triangulation to these mean points.
3. Warp each triangle from the input and target images to the corresponding mean triangles.
4. Average the pixel values of the warped triangles.

The result is below for a few images. Obama proved harder to get right as his facial structure is pretty different than mine. It looks better if you sit further away, maybe the lower frequencies match up better.

The Morph Sequence

Calling the above method in a loop with different interpolation parameters produces a series of intermediate morphs that can be made into frames. Two such morph sequences are posted here

The "Mean Face" of a Population

For this I used the Danish computer scientist dataset. The result of morphing a few Danes into the mean image is below.

In both cases there is a warping artifact on the side of the image. This is likely because I had to add a few control points to the dataset, but didn't add many on the background.

After performing this same operation on all the computer scientists (in the facing forward, smiling position) and averaging their pixels, the average face turns out to be

I also tried warping my face into the average Danish geometry. The keypoints were a little difficult to label around the nose and eyes (because there are a lot of them in close succession), so those areas may be slightly suboptimal. Overall, it appears I may not be Danish.

Finally, here is a picture of a danish computer scientist warped to my geometry.

As you can see, this warping resembles my geometry more. This is most noticable in jawline.

Caricatures: Extrapolating from the mean

To produce a caricature, I extrapolate via the following formula: $$X + \alpha(Y - X)$$ where $X$ is my geometry, $Y$ is the average Danish geometry, and we let $\alpha > 1$. The result is below, with $\alpha = 1.3$

Bells & Whistles

The first bells & whistles I did was a class morph with 8 other classmates. The video is below, my face is at the end.

Note: There is a gap in the middle because someone in our group hadn't finished their morph by the time I had to upload the video.

I also made two videos morphing across age, these are below. The second one is me morphing to my grandpa!