Face Morphing

Kevin Lin, klinime@berkeley.edu

Table of Content

Introduction

The purpose of this project is to produce morph animations that create natural transitions of a face to another. The general idea is to first triangulate images and linearly interpolate between each triangle by warping (spatial correspondence) and cross-dissolving (pixel correspondence).

Part 1, Defining Correspondences

We need to first define keypoints as vertices for triangulation. The quality of the keypoints will dictate the quality of the morph. In general, we want the important features as our keypoints. We can then triangulate with Delaunay Triangulation and produce the following:
Imgur

Part 2, Computing the “Mid-Way” Face

This is the core of our algorithm. We will use one triangulation and apply the same to all other images. Then, the general algorithm is as follows:

  1. Linearly interpolate between triangles with y=(1α)x0+αx1,α[0,1]
  2. Solve for the linear equations of y=xA where y is the original triangles, x is the interpolated, and A is our inverse transformation.
  3. For each interpolated triangle, inverse transform to find the corresponding original triangle, and perform Bilinear Interpolation to compute their corresponding pixel values. Do this on both original images to get morphed img0 and morphed img1.
  4. Linearly interpolate the pixel values between morphed img0 and morphed img1.

For the “mid-way” face, simply set alpha=0.5. Example outputs:
Imgur
Where the top row is combined naively, and the bottom row is warped and combined.

Part 3, Morph Sequence

Simply stack interpolated faces with increasing alpha from 0 to 1 will result in smooth morphing.
Imgur
Wish I can be as badass as gilgamesh lol
Imgur
Enjoy the boomerang!

Part 4, “Mean Face”

I used the FEI Face Database (example below) to compute the mean face and keypoints
Imgur
where the color and lines illustrate the order of the keypoints. The computed mean faces:
ImgurImgur
ImgurImgur
In order, they are male_neutral, male_smile, female_neutral, and female_smile.
Morphing an example to the male_neutral geometry:
Imgur
Morphing my face to the male_smile geometry:
Imgur
Morphing the male_smile geometry to my face:
Imgur

Part 5, Caricatures: Extrapolating from the Mean

By using alpha values for shapes outside of range [0, 1], we get exaggerating caricatures.
Imgur
Imgur
The above are alpha=-0.5 and alpha=1.5, yielding a skinny and a fat version of me.

Part 6 (Bells and Whistles), Changing Gender / Expression

Previously, we have two data points, my face and the mean face, and we interpolate / extrapolate linearly along the two points. Here, we have two data points we consider as templates (male/female, neutral/smile), compute the transformation from one to another, and apply the computed transformation to a new image (my face). Thus, we have my face as the base but with changing gender/expression. The results are below:
Imgur
Some notible difference for when changing gender are 1. whiter face, 2. longer hair, 3. bigger smile. Apparently girls generally have brighter smiles than guys 🤣
Imgur
A cool detail is that not only does the corners of my mouth change between the expressions, the nearby muscles also get adjusted accordingly, making the transition more natural.

Part 7 (Bells and Whistles), PCA as Faces

Similar to the previous part, we have a pair of templates to compute the transformation on. The difference is that the templates here are the mean face and a PCA face, either based on appearance or shape. The PCA is computed across all smiling faces. The following, in order, are my face, mean face, PCA vector, the morphed face, all warped to the same geometry.
Imgur
Above is the result of dissolving the PCA face for appearance. You can see that the PCA face appears to put emphasis on the smile, particularly the teeth and the muscles around the corner of the mouth. So the morphed image seems to have the smile enhanced.
Imgur
Above is the result of warping the PCA vector for shape. You can see that the PCA vector focuses on the width of the face, so when warped, my face looks skinnier.

Part 8 (Bells and Whistles), Music Video (Fate/Grand Order)

Fun with the numerous Arturia faces in FGO! Click to watch on YouTube.