Justin
D. Norman Project 3: Face Morphing!
Citation and Inspiration
Learn OpenCV
– Face Morph Using OpenCV
Overview
The goal of Project 3 is to produce a
"morph" animation of your face into someone else's face, compute the
mean of a population of faces and extrapolate from a population mean to create
a caricature of yourself.
1. Defining Correspondences
In this first section we were asked to define corresponding keypoints for two images. In order to provide a good
representation of the key areas of each face, I chose 64 points spread across
distinct features. I also wrote a gui for selecting
the points and stored them for later use, provided the correct parameter was
set. This way the (somewhat tedious) process (usually) only needed to be
completed once.
From there, it was necessary to compute a triangulation to be used as a map for
the morphing step. I took the advice of the spec, and
computed a triangulation at the midway shape of the two images. The
results of that step, displayed on both images is below:
2. Computing the “Mid-Way”
Face
The
next step was to compute the mid-way face of the two images. This consists of several steps:
1. Computing the average shape
2. Warping both faces into that
shape
3. Ensuring that the colors are
correctly averaged together
I
first constructed the affine matrix according to this approach (math.stackexchange)
The
next step was to perform an inverse warp the points of the one image into another
shape using the triangulation computed earlier.
Then I used numpy’s linear algebra functionality
to compute the morph, the result is below:
Horrifying.
However, this gave me the intuition necessary
to move on to the next step.
3. The Morph
Sequence
In this
section I created an animation of 40 frames, where each frame is a morph of the
two images, but computed with a different weighting
factor. Some issues are present in both
the cross-dissolve (not as clear as it should be due to poor alignment and
background noise) as well as around the ears, where greater granularity of keypoints would make the transitions smoother.
4. The “Mean Face” of
a Population
Due to its excellent annotation and
organization I chose the Danes dataset. The goal of this section is as follows:
- Compute the average face
shape of the whole population or some subset of the population - say, all
the old/young/white/asian/men/women etc.
However, if you pick a subpopulation - make sure it contains enough faces
for this to be interesting.
- Morph each of the faces
in the dataset into the average shape. Show us some examples.
- Compute the average face
of the population and display it.
I
chose to create a mean face of all smiling men with facial hair in the dataset. The average triangulation looks like this:
Here
is the result of the computation on the subset of the population that I chose.
Finally,
here is the mean face!
Moving
on, you can see my face morphed to the geometry of the average smiling bearded
man and the converse:
5. Extrapolating
from the Mean: Caricatures
By
adjusting the alpha, we can create caricatures with highly exaggerated
features
6. Bells and Whistles:
Creating a Morph Music Video