short summary of what this was all about :)
Warping faces is so fun they make mirrors that do just that. So how bout let's try to recreate it
with some math and some fun photos?
In this project, I morphed between image sequences by splitting up images into a bunch of triangles
and changing them via affine transformations.
In order for us to split the image into transformable triangles, we need to find points to map features between images. Using ginput, I saved points of correspondence and performed Delaunay triangulation to create the following nets.
To correctly transform each triangle we perform 2 main operations.
1. Find the affine transformation between the triangle in the start image and the triangle in the result image
as seen in the matrix represented with values a, b, ..., above
2. Using known points in the result image, retrieve corresponding points in the start image through inverse mapping with the computed affine matrix.
Then, assign pixel values from the start image at the corresponding points to the known points in the result image.
After performing the warp for the mid-way point, I then used weighted averages to cross dissolve between transformations to other weighted warps.
After trying out warps with my own photos, I looked in the collect averages of database images. Here is the dataset from the FEI face database in Sao Paulo, Brazil. I decided to look into the female sub-population of neutral images. We see that some transformations are very wonky for most people when we morph them to the average face shape. The average is the sharpest around the eyes and noisiest in the hair section.
We can think of the differences between the average and other faces as the difference between the distances of their correspondence points. This then gives us the ability to extrapolate beyond the data by adding or subtracting scalar multiples of the differences. In the gif below, I show just that. As you can see, when the coefficient multiplying the differnce is too high, the face ceases to be natural.
I love watching kdramas and particularly liked watching Scarlet Heart: Ryeo. They also had really cool posters, so here's a video sequence coordinated to the song: 'Wind (바람)' (Scarlet Heart: Ryeo OST, Part 11) by Jung Seung Hwan.
Themed Morph from Lily Yang on Vimeo.
The most important thing I learned from this project is that complex transformations can always be broken down into smaller parts :). I thought this was really fun and especially liked looking at faces warping in and out between frames.