Computing the "Mid-way Face"
To start the whole morphing process, I start by creating a mid-way face. First, a correspondence is established between the faces in two images: by manually selecting a set of points from the same locations of eyes, nose, mouth, cheek bones etc. from image 1 and image 2, I have a mapping of important facial features between the two faces. (as shown in the figure below)
Points - 1
Points - 2
Next, I calculate the average over the 2 sets of points -- a set of mid-way points. Compute Delaunay Triangulation over the mid-way points.
Using image 1 points, image 2 points and the Delaunay triangulation, I compute the affine transformation matrices between the image 1 triangle vertices' coordinates and image 2 triangle vertices' coordinates, of each triangle. Note that the affine transformation is computed for the inverse warping operation of transforming points from their positions on image 2 to corresponding spots on image 1.
With warp fraction = 0.5, cross-dissolve fraction = 0.5 (since we are generating the mid-way image):
Sara Wang (ME)
Billie Eilish
Sara + Billie
The morph Sequence
Setting the warp fraction and cross-dissolve fraction over (in my case) the range [1,0], I store the resulting intermediate images at each frame into a GIF file, with frame count = 45, fps = 30.
On the right is an experimentation I did using George Clooney and Matt Damon's photos. frame count = 30.
Sara -> Billie
Damon -> George
The "Mean face" of a population
For this part, my source images are - 37 images from the Danish Faces Dataset with pre-selected points.
Original Images
Faces morphed into the average shape.
Step 1: Parse the asf files, get the provided data points from the Danish Faces dataset. There are 58 annotated points for each image. The (x, y) coordinates are provided and Data points are nicely ordered. (Left Figure below)
The annotated points - data parsed from the given asf files
Delaunay triangulation computed on the mean set of points
Step 2: Compute the average shape of all Danish Faces, by averaging over all sets of data points in the given data set. The resulting image of the average data points is the mean shape of the population. (shown in figure below) Over the average data points, generate Delaunay triangulation (shown in the right figure above), which is used for all morph processes later.
"the average face shape of the whole population"
Step 3: Morph all Danish faces into the average face shape generated above (warp fraction = 0, cross-dissolve fraction = 1). Then compute the average of all morphed Danish faces. The resulting image is the average face of the population.
"the average face of the population"
Shown below: 1) My face warped into the average geometry, and 2) the average face warped into my geometry.
"my face warped into the average geometry"
"the average face warped into my geometry."
Caricatures: Extrapolating from the mean
Caricatures are produced by extrapolating from the population mean: take the difference between the average face of Danish population (computed in previous part) and my face, multiplied by a factor (alpha α), before adding back to my face.
e.g. alpha = 1.2, P + 1.2(Q-P), -0.2P + 1.2Q
alpha = -1.0
alpha = 1.2
alpha = 2.0
Bells and Whistles: Change the gender/ethnicity of my face
"Change age/gender/ethnicity/smile/etc of your (or your friend's) face. You can use average images off the web for this, no need to recompute the averages yourself (unless you want to). Show morphing just the shape, just the appearance, and both."
- Average face of populations -
englishman
englishwoman
irishmale
irishfemale
spanishmale
spanishfemale
- Morphed shape -
englishman
englishwoman
irishmale
irishfemale
spanishmale
spanishfemale
- Morphed appearance -
englishman
englishwoman
irishmale
irishfemale
spanishmale
spanishfemale
- Morphed both shape and appearance -
englishman
englishwoman
irishmale
irishfemale
spanishmale
spanishfemale