Face Morphing
Kevin Lin, klinime@berkeley.edu
Table of Content
Introduction
The purpose of this project is to produce morph animations that create natural transitions of a face to another. The general idea is to first triangulate images and linearly interpolate between each triangle by warping (spatial correspondence) and cross-dissolving (pixel correspondence).
Part 1, Defining Correspondences
We need to first define keypoints as vertices for triangulation. The quality of the keypoints will dictate the quality of the morph. In general, we want the important features as our keypoints. We can then triangulate with Delaunay Triangulation and produce the following:
Part 2, Computing the “Mid-Way” Face
This is the core of our algorithm. We will use one triangulation and apply the same to all other images. Then, the general algorithm is as follows:
- Linearly interpolate between triangles with y=(1−α)x0+αx1,α∈[0,1]
- Solve for the linear equations of y=xA where y is the original triangles, x is the interpolated, and A is our inverse transformation.
- For each interpolated triangle, inverse transform to find the corresponding original triangle, and perform Bilinear Interpolation to compute their corresponding pixel values. Do this on both original images to get morphed img0 and morphed img1.
- Linearly interpolate the pixel values between morphed img0 and morphed img1.
For the “mid-way” face, simply set alpha=0.5. Example outputs:
Where the top row is combined naively, and the bottom row is warped and combined.
Part 3, Morph Sequence
Simply stack interpolated faces with increasing alpha from 0 to 1 will result in smooth morphing.
Wish I can be as badass as gilgamesh lol
Enjoy the boomerang!
Part 4, “Mean Face”
I used the FEI Face Database (example below) to compute the mean face and keypoints
where the color and lines illustrate the order of the keypoints. The computed mean faces:
In order, they are male_neutral, male_smile, female_neutral, and female_smile.
Morphing an example to the male_neutral geometry:
Morphing my face to the male_smile geometry:
Morphing the male_smile geometry to my face:
By using alpha values for shapes outside of range [0, 1], we get exaggerating caricatures.
The above are alpha=-0.5 and alpha=1.5, yielding a skinny and a fat version of me.
Part 6 (Bells and Whistles), Changing Gender / Expression
Previously, we have two data points, my face and the mean face, and we interpolate / extrapolate linearly along the two points. Here, we have two data points we consider as templates (male/female, neutral/smile), compute the transformation from one to another, and apply the computed transformation to a new image (my face). Thus, we have my face as the base but with changing gender/expression. The results are below:
Some notible difference for when changing gender are 1. whiter face, 2. longer hair, 3. bigger smile. Apparently girls generally have brighter smiles than guys 🤣
A cool detail is that not only does the corners of my mouth change between the expressions, the nearby muscles also get adjusted accordingly, making the transition more natural.
Part 7 (Bells and Whistles), PCA as Faces
Similar to the previous part, we have a pair of templates to compute the transformation on. The difference is that the templates here are the mean face and a PCA face, either based on appearance or shape. The PCA is computed across all smiling faces. The following, in order, are my face, mean face, PCA vector, the morphed face, all warped to the same geometry.
Above is the result of dissolving the PCA face for appearance. You can see that the PCA face appears to put emphasis on the smile, particularly the teeth and the muscles around the corner of the mouth. So the morphed image seems to have the smile enhanced.
Above is the result of warping the PCA vector for shape. You can see that the PCA vector focuses on the width of the face, so when warped, my face looks skinnier.
Part 8 (Bells and Whistles), Music Video (Fate/Grand Order)
Fun with the numerous Arturia faces in FGO! Click to watch on YouTube.
Face Morphing
Kevin Lin, klinime@berkeley.edu
Table of Content
Introduction
The purpose of this project is to produce morph animations that create natural transitions of a face to another. The general idea is to first triangulate images and linearly interpolate between each triangle by warping (spatial correspondence) and cross-dissolving (pixel correspondence).
Part 1, Defining Correspondences
We need to first define keypoints as vertices for triangulation. The quality of the keypoints will dictate the quality of the morph. In general, we want the important features as our keypoints. We can then triangulate with Delaunay Triangulation and produce the following:
Part 2, Computing the “Mid-Way” Face
This is the core of our algorithm. We will use one triangulation and apply the same to all other images. Then, the general algorithm is as follows:
For the “mid-way” face, simply set alpha=0.5. Example outputs:
Where the top row is combined naively, and the bottom row is warped and combined.
Part 3, Morph Sequence
Simply stack interpolated faces with increasing alpha from 0 to 1 will result in smooth morphing.
Wish I can be as badass as gilgamesh lol
Enjoy the boomerang!
Part 4, “Mean Face”
I used the FEI Face Database (example below) to compute the mean face and keypoints
where the color and lines illustrate the order of the keypoints. The computed mean faces:
In order, they are male_neutral, male_smile, female_neutral, and female_smile.
Morphing an example to the male_neutral geometry:
Morphing my face to the male_smile geometry:
Morphing the male_smile geometry to my face:
Part 5, Caricatures: Extrapolating from the Mean
By using alpha values for shapes outside of range [0, 1], we get exaggerating caricatures.
The above are alpha=-0.5 and alpha=1.5, yielding a skinny and a fat version of me.
Part 6 (Bells and Whistles), Changing Gender / Expression
Previously, we have two data points, my face and the mean face, and we interpolate / extrapolate linearly along the two points. Here, we have two data points we consider as templates (male/female, neutral/smile), compute the transformation from one to another, and apply the computed transformation to a new image (my face). Thus, we have my face as the base but with changing gender/expression. The results are below:
Some notible difference for when changing gender are 1. whiter face, 2. longer hair, 3. bigger smile. Apparently girls generally have brighter smiles than guys 🤣
A cool detail is that not only does the corners of my mouth change between the expressions, the nearby muscles also get adjusted accordingly, making the transition more natural.
Part 7 (Bells and Whistles), PCA as Faces
Similar to the previous part, we have a pair of templates to compute the transformation on. The difference is that the templates here are the mean face and a PCA face, either based on appearance or shape. The PCA is computed across all smiling faces. The following, in order, are my face, mean face, PCA vector, the morphed face, all warped to the same geometry.
Above is the result of dissolving the PCA face for appearance. You can see that the PCA face appears to put emphasis on the smile, particularly the teeth and the muscles around the corner of the mouth. So the morphed image seems to have the smile enhanced.
Above is the result of warping the PCA vector for shape. You can see that the PCA vector focuses on the width of the face, so when warped, my face looks skinnier.
Part 8 (Bells and Whistles), Music Video (Fate/Grand Order)
Fun with the numerous Arturia faces in FGO! Click to watch on YouTube.