Does anyone remember that children's series Animorphs? This is the computer vision version of that.
Defining Correspondences
For the first step, I defined a set of points in 2 images, where one point in the first image would correspond to another point in the second image. I use these points to create a Delaunay triangulation, the idea here being that if we polygonize the images and apply an affine transformation on each polygon separately, we will get a smooth morph from one image to another, with key visual points (such as eyes, nose, and mouth) connected together. Below is the Delaunay triangulation that resulted from my points (I took the average between the two sets of points to produce the points used for the triangulation).
|
Computing The "Mid-Way Face"
To compute the midway face, I proceeded to perform the affine transformation on each triangle. I get the affine by performing the dot product between the midway shape points and image 1's (me) points. I then inverse it, and apply that inverse affine to the midway shape to get the appropriate indices to image 1. This allows me to populate each pixel in the midway image without any gaps created by float rounding after computing the affine transformation. I also blend the colors of me and the target image (Mr. Sakurai, Lord of Smash) by also computing the affine transformation between the midway shape and image 2, and then use that to get the appropriate pixels to sample from image 2. The result is shown below:
|
The Morph Sequence
To create the morph sequence, I basically perform the same operation as above, except with a different warp ratio. The warp ratio will range from 0 to 1, at increments of x/45 (since we have 45 frames). The color weighting will also be modified to reflect this warp ratio. Thus, we can compile the 45 frames we get and get a morph sequence, played back at roughly 30 frames per second.
|
The Mean Face of a Population
I use images from the FEI face database in order to compute the average face. I took the 200 frontal faces (non-smiling) and got the average shape using the same techniques as above (shown below).
|
I then warped those faces to this average shape, again using the same technique as I did for the midway face. These are some examples of the results, done on face 6, 42, and 135.
|
|
|
I then compute an average face by taking the first 50 faces (not 200, because that takes too long and I am once again asking for time from the universe!), and average the color value as well. This is the result!
|
|
|
Caricatures: Extrapolating from the Mean
To make a caricature, I morph myself into the average face's geometry, but spike the morph factor to be greater than 1. This enhances the warp and makes it go farther than the original geometry intends, creating a warped, extreme image.
|
Bells and Whistles: A Music Video!
I basically repeated performed the morph sequence between different images from the video game Super Smash Bros. (specifically the Fire Emblem characters) and made a music video where they all morph into each other! The video can be found here.