Getting Started
The package is written in Python 3 and requires a recent version of numpy
, scipy
, and scikit-image
. Invoke the main.py
file to run each tool.
main.py
contains several different sub-utilities for defining input
correspondence points, morph
ing one shape into another, computing the mean
of a population, an optimized procedure for applying a shape-warp
without a cross-dissolve blend, and a method for extrapolating a caricature
based on existing geometry.
usage: main.py [-h] {input,morph,mean,warp,caricature} ...
Utilities for face morphing
positional arguments:
{input,morph,mean,warp,caricature}
Compute face morphs
input Input data points
morph Morph one face to another
mean Compute the mean face
warp Warp shape to geometry
caricature Extrapolate from the mean shape
optional arguments:
-h, --help show this help message and exit
Defining Correspondences
Cross-dissolves are fairly straightforward: at each pixel, we just need to take a weighted proportion of each image. The challenge, then, is to come up with the correct change in shape.
In order to define a change in shape, we must first identify the shapes themselves. Because most real-world samples are not simple enough to solve using a global transformation for each pixel, so we'd like to be able to define localized transformations on each part of the input image. But defining the localized mesh grid becomes a problem in its own as it's difficult to define each change in the dense grid.
We instead use a sparser representation based on correspondence points to capture matching features between the two subjects. Landmarks are used to annotate each subject's features to obtain a one-to-one mapping of each feature point. Then, we generate the mesh grid from the correspondence points by employing a Delaunay triangulation across the points.
Mid-Way Face
In order to produce the full morph procedure, we first needed to define a function which could compute each intermediary step of the animation. This "mid-way face" is computed by determining the affine transformation between the points of each triangle from the source image to the destination image.
The affine transformation can then be applied to 'pluck' the right colors from the corresponding points in the source and the destination. This captures the shape of the mid-way face. To make it believable, we finally cross-dissolve each pixel to blend the two warped shapes together.
Morph Sequence
The mid-way face represents a single frame in the final animation, so we can create the full animation by computing the mid-way face for each timestep between the source image and the destination image.
"Mean Face"
We then extended with the mean
command, which allows us to compute the average over not just two images, but any arbitrarily-large set of images. This allows us to generate interesting visualizations of an entire population of images.
Caricatures
We can also generate caricatures by extrapolating shapes to and from the mean. Instead of warping between two images, we start with the input, and stretch the transform to exaggerate the image.
Here is a collection of images extrapolating toward the mean face shape of the Brazilian FEI database.
Here is a collection of images extrapolating away from the mean face shape of the Brazilian FEI database.
Shape and Appearance
We can also reimagine our own faces using face morphing. Instead of warping between faces, if we hold the appearance or shape constant while modifying the other variable, we can reimagine our faces under in a different gender, ethnicity, or age.
We first collected a sample of three average face images found online.
Using the same procedure above, we generated correspondence points. Here, we show the correspondence points triangulated over my face.
We can then observe the outcome as we hold the appearance constant, but warping the shape into my face.
Or, if we hold the shape constant but dissolve the corresponding colors into my face.
We can also morph both shape and appearance at the same time, similar to the morph procedures we've demonstrated previously.
Using the same technique, we can also generate animations of the morphing process using both appearance and shape.