To perform a face morph, the first step is to define corresponding points on the images and produce a triangulation. We use cpselect to pick the points by hand and generate the list of points. Then, we calculate the Delaunay triangulation based on the average between the two sets of points to avoid potential triangle deformations. Here is the triangulation applied to points on both images:
|
|
Now that we have the corresponding points and triangulation on the two faces, we need to warp the two images into the average shape, then average over the colors to compute the mid-way face. To help with computing the warp, we define computeAffine(tri1_pts,tri2_pts), which will compute the affine transformation from tri1 to tri2. We will then, for a given image, use the affine transformation to compute the inverse warp by repeating the following on every triangle:
Compute the affine transform from image triangle to the average triangle and take the inverse.
Find the pixel coordinates that correspond to the average triangle coordinates using skimage.draw.polygon.
Apply the inverse affine transform to the average pixel coordinates to get the image pixel coordinates.
Finally, we set the values at the average pixel coordinates to equal the values at the corresponding image pixel coordinates.
That gives us an image warped to the shape of the average. So, we do this for both images so they fit the average shape, then simply average over the pixels to obtain the mid-way face.
|
|
|
Now, we want to create a generalized version of the mid-way face, but now with the ability to control how much we warp the shapes and how much we dissolve the pixels. We define morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac) to produce this result. The process is very similar to the one for the mid-way face, but with two key differences:
When calculating the points to define the shape of the morph, instead of taking the average, we weight the points such that we take warp_frac of im1_pts and (1-warp_frac) of im2_pts.
When cross dissolving the pixels, instead of taking the average of all the pixels, we again weight the pixel values such that we take dissolve_frac of im1 and (1-dissolve_frac) of im2.
Note that we can now simply use the generalized morph function with warp_frac=dissolve_frac=0.5 to compute the mid-way face.
To create the morph animation, we call morph 46 times, with warp_frac=dissolve_frac advancing in equal increments from 0 to 1 to create 46 frames (0-45) of animation. On frame 0, the image shown is 100% im2 (me), and on frame 45, the image shown is 100% im1 (George). Here is the warp animation:
|
We will now calculate the mean face across the FEI face database. The data set comes with 200 smiling images and 200 neutral-faced images, as well as annotated corresponding points. All of the provided points lie direclty on the face, so for each image/set of points, I added points in each of the four corners, as well as the vertical center on the left and right sides. I calculated the average faces across all images, all smiling images, and all neutral images:
|
|
|
We can also apply a warp of each image in the data set to the average shape. Because of the smiling/neutral difference, I decided to apply the average smiling shape to smiling images and average neutral shape to neutral images. Here are some of the results:
|
|
|
|
|
|
|
|
Additionally, we can warp my face into the average shape of the data set, or we can warp the average face into the shape of my face:
|
|
We can also take my face and the population mean from the last step and extrapolate to create a caricature. In a morph of the two with dissolve fraction fixed at 0, if the warp fraction at 0 gives my original face, we now set the warp fraction to some negative value; I chose -0.5. This gives a version of my face where my features that differ from the mean face are exaggerated. We can see the result clearly as my eyes and mouth get even smaller than they already are.
|
For bells and whistles, I decided to change the gender of my face. I found images for the average Chinese man and woman (from here: https://pmsol3.wordpress.com/2009/10/10/world-of-facial-averages-east-southeast-asia-pacific-islander/) to do so.
|
|
First, I took the difference between the average woman and average man to get the "vector" that would make a man look more like a woman. Then I used the "vector" and my face and defined corresponding points, then performed half-way morphs with just shape, just color, and both shape and color
|
|
|
|