Overview
In this assignment you will produce a "morph" animation of some one's face into someone else's face, and compute the mean of a population of faces and extrapolate from a population mean to create a caricature of myself.
Defining Correspondence
To do so, for some pairs of images, we need to define the corresponding points by hand. I did this by using 'cpselect' package in python that basically does what cpselect does in matlab. Then, I computed Delauny triangulation using scipy.spatial.Delauny function, which results in triangulation like the following example.
Mid-way Face
Now with the images' corresponding points and their triangulation, we are ready to proceed. With them, we can compute the 'mid-way face' of two images. We can do this by 1) computing the average shape (a.k.a the average of each keypoint location in the two faces), 2) warping both faces into that shape, and 3) averaging the colors together.
To 'warp' an image to another, we first compute the transformation matrix that maps the vertices of a triangle to those of the corresponding triangle. That is, given [x1, y1, 1] where x1, y1 are the coordinate of a vertice and [x2, y2, 1] where x2, y2 are the coordinate of the corresponding vertex, we need to compute matrix T such that T [x1, y1, 1]^T = [x2, y2, 1] for all 3 pairs of vertices. We can simply do this by construting two 3x3 matrices A and B, each with a triangle's 3 vertices as its columns, and compute T such that TA = B. We can do this by computing the inverse of matrix A. i.e., T = BA^-1. Since not every matrix is invertible, for stability compute the pseudo-inverse.
Morph Sequence
Now, by producing multiple 'mid-way faces' by increasing the warp fraction and cross-dissolve fraction step by step (from 0 to 1), we can produce a sequence of morph.
The resulting gifs were pleasing to watch. Here are some fun results I had :)
The "Mean face" of a population
By calculating the average shape of a population (or a sub population) and taking the average of the individual images warped to the average shape, we can get the 'mean face' of the population.
From FEI Face Database, I got 14 images for each of 200 brazilian individuals, a total of 2800 images.
Here are some individual smiling images warped into the average smiling shape.
Now, by adding up all the warped images as above, we can get the mean faces. Here are the mean faces.
I can warp my face into the average shape of Brazilians to make myself look more Brazilian. Or I can also do it the other way around!
Caricatures: Extrapolating from the mean
One can create a caricature, a face with exaggerated features, by extrapolating from the mean-face. The followings are caricatures of myself I got by extrapolating from the mean-face of all the brazilian faces. One can see that my eyes, nose, mouth are centered more as alpha increases.
Bells and Whistles
I made a morphing music video of Agumon from the animation 'Digimon'. Although the morphing sequences are not perfectly natural due to differenes in orientations and shapes, I had to make this video the moment I heard about image morphing. I just had to.